[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786092453486433-4734-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:47:33 dist-test-slave-6zbq krb5kdc[4743](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:47:33 dist-test-slave-6zbq krb5kdc[4743](info): set up 2 sockets
Aug 07 08:47:33 dist-test-slave-6zbq krb5kdc[4743](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260807 08:47:34.629369 4734 mini_kdc.cc:121] Time spent starting KDC: real 1.111s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786092453486433-4734-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786092453486433-4734-0/krb5kdc/test-user.keytab.
Aug 07 08:47:34 dist-test-slave-6zbq krb5kdc[4743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092454, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:47:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:47:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.159.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.159.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.159.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab.
Entry for principal kudu/127.4.159.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab.
I20260807 08:47:34.734162 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786092453486433-4734-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786092453486433-4734-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786092453486433-4734-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.159.190:44203
--webserver_interface=127.4.159.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab
--principal=kudu/127.4.159.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:39301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.159.190:44203
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:47:34.818418 4765 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:47:34.818608 4765 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:47:34.818638 4765 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:47:34.819756 4765 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:47:34.819796 4765 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:47:34.819816 4765 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:47:34.819840 4765 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:47:34.819864 4765 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:47:34.821489 4765 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:39301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786092453486433-4734-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786092453486433-4734-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.159.190:44203
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab
--principal=kudu/127.4.159.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.159.190:44203
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.159.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4765
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786092453486433-4734-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:47:34.821878 4765 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:47:34.822118 4765 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:47:34.828923 4773 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:47:34.828918 4779 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:47:34.829384 4776 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:47:34.829531 4765 server_base.cc:1034] running on GCE node
I20260807 08:47:34.829854 4765 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:47:34.830217 4765 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:47:34.831362 4765 hybrid_clock.cc:648] HybridClock initialized: now 1786092454831364 us; error 88 us; skew 500 ppm
Aug 07 08:47:34 dist-test-slave-6zbq krb5kdc[4743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092454, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:47:34.833850 4765 init.cc:377] Logged in from keytab as kudu/127.4.159.190@KRBTEST.COM (short username kudu)
I20260807 08:47:34.834461 4765 webserver.cc:460] Webserver started at http://127.4.159.190:36901/ using document root <none> and password file <none>
I20260807 08:47:34.834724 4765 fs_manager.cc:359] Metadata directory not provided
I20260807 08:47:34.834792 4765 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:47:34.834964 4765 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:47:34.836071 4765 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786092453486433-4734-0/minicluster-data/master-0/data/instance:
uuid: "34829cbfb7fd49f191aa5d1146902510"
format_stamp: "Formatted at 2026-08-07 08:47:34 on dist-test-slave-6zbq"
server_key: "05d6b80e353a8970c03d9063aa47187a"
server_key_iv: "3b7cf38d0202bbfef11767f644b29d4d"
server_key_version: "encryptionkey@0"
I20260807 08:47:34.836392 4765 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786092453486433-4734-0/minicluster-data/master-0/wal/instance:
uuid: "34829cbfb7fd49f191aa5d1146902510"
format_stamp: "Formatted at 2026-08-07 08:47:34 on dist-test-slave-6zbq"
server_key: "05d6b80e353a8970c03d9063aa47187a"
server_key_iv: "3b7cf38d0202bbfef11767f644b29d4d"
server_key_version: "encryptionkey@0"
I20260807 08:47:34.838109 4765 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 08:47:34.839190 4788 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:34.839445 4765 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 08:47:34.839533 4765 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786092453486433-4734-0/minicluster-data/master-0/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786092453486433-4734-0/minicluster-data/master-0/wal
uuid: "34829cbfb7fd49f191aa5d1146902510"
format_stamp: "Formatted at 2026-08-07 08:47:34 on dist-test-slave-6zbq"
server_key: "05d6b80e353a8970c03d9063aa47187a"
server_key_iv: "3b7cf38d0202bbfef11767f644b29d4d"
server_key_version: "encryptionkey@0"
I20260807 08:47:34.839591 4765 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786092453486433-4734-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786092453486433-4734-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786092453486433-4734-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:47:34.854719 4765 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:47:34.856622 4765 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:47:34.856805 4765 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:47:34.861624 4765 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.190:44203
I20260807 08:47:34.861704 4857 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.190:44203 every 8 connection(s)
I20260807 08:47:34.862286 4765 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
I20260807 08:47:34.863739 4860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:47:34.867079 4860 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34829cbfb7fd49f191aa5d1146902510: Bootstrap starting.
I20260807 08:47:34.868009 4860 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 34829cbfb7fd49f191aa5d1146902510: Neither blocks nor log segments found. Creating new log.
I20260807 08:47:34.868379 4860 log.cc:824] T 00000000000000000000000000000000 P 34829cbfb7fd49f191aa5d1146902510: Log is configured to *not* fsync() on all Append() calls
I20260807 08:47:34.869143 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 4765
I20260807 08:47:34.869264 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786092453486433-4734-0/minicluster-data/master-0/wal/instance
I20260807 08:47:34.869268 4860 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34829cbfb7fd49f191aa5d1146902510: No bootstrap required, opened a new log
I20260807 08:47:34.870095 4734 external_mini_cluster.cc:1428] Setting key 2ffc92241f10a35aea17ba49806d3250
I20260807 08:47:34.871747 4860 raft_consensus.cc:348] T 00000000000000000000000000000000 P 34829cbfb7fd49f191aa5d1146902510 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34829cbfb7fd49f191aa5d1146902510" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 44203 } }
I20260807 08:47:34.871968 4860 raft_consensus.cc:374] T 00000000000000000000000000000000 P 34829cbfb7fd49f191aa5d1146902510 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:47:34.872092 4860 raft_consensus.cc:729] T 00000000000000000000000000000000 P 34829cbfb7fd49f191aa5d1146902510 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34829cbfb7fd49f191aa5d1146902510, State: Initialized, Role: FOLLOWER
I20260807 08:47:34.872273 4860 consensus_queue.cc:260] T 00000000000000000000000000000000 P 34829cbfb7fd49f191aa5d1146902510 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34829cbfb7fd49f191aa5d1146902510" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 44203 } }
I20260807 08:47:34.872475 4860 raft_consensus.cc:388] T 00000000000000000000000000000000 P 34829cbfb7fd49f191aa5d1146902510 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:47:34.872522 4860 raft_consensus.cc:482] T 00000000000000000000000000000000 P 34829cbfb7fd49f191aa5d1146902510 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:47:34.872570 4860 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 34829cbfb7fd49f191aa5d1146902510 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:47:34.873824 4860 raft_consensus.cc:504] T 00000000000000000000000000000000 P 34829cbfb7fd49f191aa5d1146902510 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34829cbfb7fd49f191aa5d1146902510" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 44203 } }
I20260807 08:47:34.874058 4860 leader_election.cc:302] T 00000000000000000000000000000000 P 34829cbfb7fd49f191aa5d1146902510 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 34829cbfb7fd49f191aa5d1146902510; no voters:
I20260807 08:47:34.874327 4860 leader_election.cc:288] T 00000000000000000000000000000000 P 34829cbfb7fd49f191aa5d1146902510 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:47:34.874377 4866 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 34829cbfb7fd49f191aa5d1146902510 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:47:34.874547 4866 raft_consensus.cc:686] T 00000000000000000000000000000000 P 34829cbfb7fd49f191aa5d1146902510 [term 1 LEADER]: Becoming Leader. State: Replica: 34829cbfb7fd49f191aa5d1146902510, State: Running, Role: LEADER
I20260807 08:47:34.874713 4866 consensus_queue.cc:237] T 00000000000000000000000000000000 P 34829cbfb7fd49f191aa5d1146902510 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34829cbfb7fd49f191aa5d1146902510" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 44203 } }
I20260807 08:47:34.874852 4860 sys_catalog.cc:564] T 00000000000000000000000000000000 P 34829cbfb7fd49f191aa5d1146902510 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:47:34.875191 4866 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34829cbfb7fd49f191aa5d1146902510 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 34829cbfb7fd49f191aa5d1146902510. Latest consensus state: current_term: 1 leader_uuid: "34829cbfb7fd49f191aa5d1146902510" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34829cbfb7fd49f191aa5d1146902510" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 44203 } } }
I20260807 08:47:34.875420 4866 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34829cbfb7fd49f191aa5d1146902510 [sys.catalog]: This master's current role is: LEADER
I20260807 08:47:34.875352 4867 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34829cbfb7fd49f191aa5d1146902510 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "34829cbfb7fd49f191aa5d1146902510" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34829cbfb7fd49f191aa5d1146902510" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 44203 } } }
I20260807 08:47:34.875680 4867 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34829cbfb7fd49f191aa5d1146902510 [sys.catalog]: This master's current role is: LEADER
I20260807 08:47:34.876520 4884 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:47:34.877835 4884 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:47:34.880182 4884 catalog_manager.cc:1347] Generated new cluster ID: 8906d723a1724dd9bcb34405700be831
I20260807 08:47:34.880232 4884 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 08:47:34 dist-test-slave-6zbq krb5kdc[4743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092454, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.159.190@KRBTEST.COM
I20260807 08:47:34.890129 4864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:34.871512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56946 (local address 127.4.159.190:44203)
0807 08:47:34.871832 (+ 320us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:34.871843 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:34.872037 (+ 194us) server_negotiation.cc:407] Connection header received
0807 08:47:34.875552 (+ 3515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:34.875590 (+ 38us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:34.875721 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:34.875841 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:47:34.878129 (+ 2288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:34.879267 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:34.880778 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:34.881406 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:34.881500 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:34.885786 (+ 4286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:47:34.885791 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:47:34.885791 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:47:34.885803 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:47:34.887843 (+ 2040us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:34.888297 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:34.888301 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:34.888301 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:34.888441 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:34.888691 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:34.888692 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:34.888692 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:34.889036 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:47:34.889274 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:34.889443 (+ 169us) server_negotiation.cc:299] Negotiation successful
0807 08:47:34.889620 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":94,"threads_started":1}
I20260807 08:47:34.906834 4884 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:47:34.907600 4884 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:47:34.913025 4884 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 34829cbfb7fd49f191aa5d1146902510: Generated new TSK 0
I20260807 08:47:34.913203 4884 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.159.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.159.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.159.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.129.keytab.
Entry for principal kudu/127.4.159.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.129.keytab.
I20260807 08:47:34.942140 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786092453486433-4734-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786092453486433-4734-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786092453486433-4734-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786092453486433-4734-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.159.129:0
--local_ip_for_outbound_sockets=127.4.159.129
--webserver_interface=127.4.159.129
--webserver_port=0
--tserver_master_addrs=127.4.159.190:44203
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.129.keytab
--principal=kudu/127.4.159.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:39301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:47:35.038022 4905 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:47:35.038210 4905 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:47:35.038244 4905 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:47:35.039403 4905 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:47:35.039443 4905 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:47:35.039479 4905 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.159.129
I20260807 08:47:35.040998 4905 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:39301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786092453486433-4734-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786092453486433-4734-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.129.keytab
--principal=kudu/127.4.159.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.159.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786092453486433-4734-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.159.129
--webserver_port=0
--tserver_master_addrs=127.4.159.190:44203
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4905
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.159.129
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786092453486433-4734-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:47:35.041306 4905 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:47:35.041503 4905 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:47:35.047003 4918 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:47:35.047034 4914 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:47:35.047336 4913 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:47:35.047542 4905 server_base.cc:1034] running on GCE node
I20260807 08:47:35.047711 4905 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:47:35.047919 4905 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:47:35.049077 4905 hybrid_clock.cc:648] HybridClock initialized: now 1786092455049055 us; error 36 us; skew 500 ppm
Aug 07 08:47:35 dist-test-slave-6zbq krb5kdc[4743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092455, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:47:35.051800 4905 init.cc:377] Logged in from keytab as kudu/127.4.159.129@KRBTEST.COM (short username kudu)
I20260807 08:47:35.052309 4905 webserver.cc:460] Webserver started at http://127.4.159.129:36999/ using document root <none> and password file <none>
I20260807 08:47:35.052568 4905 fs_manager.cc:359] Metadata directory not provided
I20260807 08:47:35.052644 4905 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:47:35.052781 4905 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:47:35.054019 4905 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786092453486433-4734-0/minicluster-data/ts-0/data/instance:
uuid: "c70c030d78f14f5b8155a8a9b0f8c05a"
format_stamp: "Formatted at 2026-08-07 08:47:35 on dist-test-slave-6zbq"
server_key: "a16286ee83d397dc2fbe9b103fd08f19"
server_key_iv: "98fb0fa081d52b139a7b2bcff359a483"
server_key_version: "encryptionkey@0"
I20260807 08:47:35.054420 4905 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786092453486433-4734-0/minicluster-data/ts-0/wal/instance:
uuid: "c70c030d78f14f5b8155a8a9b0f8c05a"
format_stamp: "Formatted at 2026-08-07 08:47:35 on dist-test-slave-6zbq"
server_key: "a16286ee83d397dc2fbe9b103fd08f19"
server_key_iv: "98fb0fa081d52b139a7b2bcff359a483"
server_key_version: "encryptionkey@0"
I20260807 08:47:35.056066 4905 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:47:35.056913 4927 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:35.057139 4905 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:47:35.057227 4905 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786092453486433-4734-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786092453486433-4734-0/minicluster-data/ts-0/wal
uuid: "c70c030d78f14f5b8155a8a9b0f8c05a"
format_stamp: "Formatted at 2026-08-07 08:47:35 on dist-test-slave-6zbq"
server_key: "a16286ee83d397dc2fbe9b103fd08f19"
server_key_iv: "98fb0fa081d52b139a7b2bcff359a483"
server_key_version: "encryptionkey@0"
I20260807 08:47:35.057299 4905 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786092453486433-4734-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786092453486433-4734-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786092453486433-4734-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:47:35.065480 4905 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:47:35.067178 4905 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:47:35.067298 4905 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:47:35.067530 4905 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:47:35.067866 4905 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:47:35.067950 4905 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:35.067991 4905 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:47:35.068011 4905 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:35.073810 4905 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.129:45251
I20260807 08:47:35.073851 5094 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.129:45251 every 8 connection(s)
I20260807 08:47:35.074115 4905 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786092453486433-4734-0/minicluster-data/ts-0/data/info.pb
I20260807 08:47:35.077107 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 4905
I20260807 08:47:35.077188 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786092453486433-4734-0/minicluster-data/ts-0/wal/instance
I20260807 08:47:35.077335 4734 external_mini_cluster.cc:1428] Setting key 8b48acc4a9f9bdf60594b13a15faa533
Aug 07 08:47:35 dist-test-slave-6zbq krb5kdc[4743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092455, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.129@KRBTEST.COM for kudu/127.4.159.190@KRBTEST.COM
I20260807 08:47:35.083664 4864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:35.074926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.129:46379 (local address 127.4.159.190:44203)
0807 08:47:35.074969 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:35.074971 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:35.075179 (+ 208us) server_negotiation.cc:407] Connection header received
0807 08:47:35.075585 (+ 406us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:35.075586 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:35.075596 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:35.075640 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:47:35.076648 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:35.077180 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:35.078476 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:35.078663 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:35.078718 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:35.080982 (+ 2264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:47:35.080983 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:47:35.080983 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:47:35.080990 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:47:35.082299 (+ 1309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:35.082761 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:35.082761 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:35.082762 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:35.082809 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:35.083065 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:35.083065 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:35.083065 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:35.083163 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:47:35.083209 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:35.083526 (+ 317us) server_negotiation.cc:299] Negotiation successful
0807 08:47:35.083593 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 08:47:35.083895 5097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:35.074899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:44203 (local address 127.4.159.129:46379)
0807 08:47:35.075063 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0807 08:47:35.075103 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 08:47:35.075515 (+ 412us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:47:35.075749 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:47:35.075753 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:47:35.076389 (+ 636us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:47:35.076569 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:35.076573 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:35.077285 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:35.077286 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:35.078401 (+ 1115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:35.078405 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:35.078738 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:35.078739 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:35.078862 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:35.079281 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:47:35.079291 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:47:35.080897 (+ 1606us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:47:35.082394 (+ 1497us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:47:35.082402 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:47:35.082404 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:47:35.082689 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:47:35.082869 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:47:35.082869 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:47:35.082870 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:47:35.083005 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:47:35.083219 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:47:35.083225 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:47:35.083398 (+ 173us) client_negotiation.cc:769] Sending connection context
0807 08:47:35.083517 (+ 119us) client_negotiation.cc:240] Negotiation successful
0807 08:47:35.083679 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":71,"threads_started":1}
I20260807 08:47:35.084425 5095 heartbeater.cc:347] Connected to a master server at 127.4.159.190:44203
I20260807 08:47:35.084523 5095 heartbeater.cc:464] Registering TS with master...
I20260807 08:47:35.084707 5095 heartbeater.cc:511] Master 127.4.159.190:44203 requested a full tablet report, sending...
I20260807 08:47:35.085647 4810 ts_manager.cc:194] Registered new tserver with Master: c70c030d78f14f5b8155a8a9b0f8c05a (127.4.159.129:45251)
WARNING: no policy specified for kudu/127.4.159.130@KRBTEST.COM; defaulting to no policy
I20260807 08:47:35.086418 4810 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.159.129@KRBTEST.COM'} at 127.4.159.129:46379
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.159.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.159.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.130.keytab.
Entry for principal kudu/127.4.159.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.130.keytab.
I20260807 08:47:35.100476 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786092453486433-4734-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786092453486433-4734-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786092453486433-4734-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786092453486433-4734-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.159.130:0
--local_ip_for_outbound_sockets=127.4.159.130
--webserver_interface=127.4.159.130
--webserver_port=0
--tserver_master_addrs=127.4.159.190:44203
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.130.keytab
--principal=kudu/127.4.159.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:39301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:47:35.178712 5100 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:47:35.178896 5100 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:47:35.178928 5100 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:47:35.180013 5100 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:47:35.180051 5100 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:47:35.180085 5100 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.159.130
I20260807 08:47:35.181493 5100 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:39301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786092453486433-4734-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786092453486433-4734-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.130.keytab
--principal=kudu/127.4.159.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.159.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786092453486433-4734-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.159.130
--webserver_port=0
--tserver_master_addrs=127.4.159.190:44203
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5100
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.159.130
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786092453486433-4734-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:47:35.181718 5100 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:47:35.181924 5100 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:47:35.186307 5112 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:47:35.186357 5108 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:47:35.186476 5100 server_base.cc:1034] running on GCE node
W20260807 08:47:35.186309 5109 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:47:35.186703 5100 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:47:35.186936 5100 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:47:35.188072 5100 hybrid_clock.cc:648] HybridClock initialized: now 1786092455188058 us; error 27 us; skew 500 ppm
Aug 07 08:47:35 dist-test-slave-6zbq krb5kdc[4743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092455, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:47:35.190274 5100 init.cc:377] Logged in from keytab as kudu/127.4.159.130@KRBTEST.COM (short username kudu)
I20260807 08:47:35.190696 5100 webserver.cc:460] Webserver started at http://127.4.159.130:46429/ using document root <none> and password file <none>
I20260807 08:47:35.190881 5100 fs_manager.cc:359] Metadata directory not provided
I20260807 08:47:35.190919 5100 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:47:35.191004 5100 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:47:35.191872 5100 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786092453486433-4734-0/minicluster-data/ts-1/data/instance:
uuid: "5e54ab2b308141439af9dba657488551"
format_stamp: "Formatted at 2026-08-07 08:47:35 on dist-test-slave-6zbq"
server_key: "5d180e5526a7373286cf0c295b2257ab"
server_key_iv: "ece7805bd4c564d0a9545c0e114c1b15"
server_key_version: "encryptionkey@0"
I20260807 08:47:35.192241 5100 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786092453486433-4734-0/minicluster-data/ts-1/wal/instance:
uuid: "5e54ab2b308141439af9dba657488551"
format_stamp: "Formatted at 2026-08-07 08:47:35 on dist-test-slave-6zbq"
server_key: "5d180e5526a7373286cf0c295b2257ab"
server_key_iv: "ece7805bd4c564d0a9545c0e114c1b15"
server_key_version: "encryptionkey@0"
I20260807 08:47:35.193429 5100 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 08:47:35.194041 5120 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:35.194185 5100 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:35.194269 5100 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786092453486433-4734-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786092453486433-4734-0/minicluster-data/ts-1/wal
uuid: "5e54ab2b308141439af9dba657488551"
format_stamp: "Formatted at 2026-08-07 08:47:35 on dist-test-slave-6zbq"
server_key: "5d180e5526a7373286cf0c295b2257ab"
server_key_iv: "ece7805bd4c564d0a9545c0e114c1b15"
server_key_version: "encryptionkey@0"
I20260807 08:47:35.194320 5100 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786092453486433-4734-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786092453486433-4734-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786092453486433-4734-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:47:35.213050 5100 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:47:35.214591 5100 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:47:35.214689 5100 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:47:35.214834 5100 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:47:35.215080 5100 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:47:35.215111 5100 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:35.215139 5100 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:47:35.215163 5100 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:35.220330 5100 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.130:41443
I20260807 08:47:35.220404 5300 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.130:41443 every 8 connection(s)
I20260807 08:47:35.220621 5100 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786092453486433-4734-0/minicluster-data/ts-1/data/info.pb
I20260807 08:47:35.224490 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 5100
I20260807 08:47:35.224581 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786092453486433-4734-0/minicluster-data/ts-1/wal/instance
I20260807 08:47:35.224737 4734 external_mini_cluster.cc:1428] Setting key 7732247f0c8d1d18ace5260371087d81
Aug 07 08:47:35 dist-test-slave-6zbq krb5kdc[4743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092455, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.130@KRBTEST.COM for kudu/127.4.159.190@KRBTEST.COM
I20260807 08:47:35.229352 4864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:35.221241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.130:44137 (local address 127.4.159.190:44203)
0807 08:47:35.221295 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:35.221298 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:35.221466 (+ 168us) server_negotiation.cc:407] Connection header received
0807 08:47:35.221891 (+ 425us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:35.221892 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:35.221905 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:35.221956 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:47:35.222935 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:35.223440 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:35.224354 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:35.224646 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:35.224728 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:35.226896 (+ 2168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:47:35.226898 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:47:35.226898 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:47:35.226905 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:47:35.228101 (+ 1196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:35.228462 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:35.228463 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:35.228464 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:35.228521 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:35.228739 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:35.228740 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:35.228740 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:35.228853 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:47:35.228908 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:35.229161 (+ 253us) server_negotiation.cc:299] Negotiation successful
0807 08:47:35.229259 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260807 08:47:35.229465 5307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:35.221215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:44203 (local address 127.4.159.130:44137)
0807 08:47:35.221359 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0807 08:47:35.221391 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0807 08:47:35.221793 (+ 402us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:47:35.222018 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:47:35.222021 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:47:35.222551 (+ 530us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:47:35.222819 (+ 268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:35.222825 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:35.223523 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:35.223523 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:35.224267 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:35.224269 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:35.224736 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:35.224737 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:35.224867 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:35.225284 (+ 417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:47:35.225294 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:47:35.226815 (+ 1521us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:47:35.228174 (+ 1359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:47:35.228181 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:47:35.228183 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:47:35.228393 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:47:35.228574 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:47:35.228574 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:47:35.228574 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:47:35.228683 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:47:35.228906 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:47:35.228912 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:47:35.229046 (+ 134us) client_negotiation.cc:769] Sending connection context
0807 08:47:35.229130 (+ 84us) client_negotiation.cc:240] Negotiation successful
0807 08:47:35.229269 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":68,"threads_started":1}
I20260807 08:47:35.229928 5301 heartbeater.cc:347] Connected to a master server at 127.4.159.190:44203
I20260807 08:47:35.230002 5301 heartbeater.cc:464] Registering TS with master...
I20260807 08:47:35.230155 5301 heartbeater.cc:511] Master 127.4.159.190:44203 requested a full tablet report, sending...
I20260807 08:47:35.230530 4810 ts_manager.cc:194] Registered new tserver with Master: 5e54ab2b308141439af9dba657488551 (127.4.159.130:41443)
I20260807 08:47:35.230875 4810 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.159.130@KRBTEST.COM'} at 127.4.159.130:44137
WARNING: no policy specified for kudu/127.4.159.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.159.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.159.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.131.keytab.
Entry for principal kudu/127.4.159.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.131.keytab.
I20260807 08:47:35.246399 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786092453486433-4734-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786092453486433-4734-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786092453486433-4734-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786092453486433-4734-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.159.131:0
--local_ip_for_outbound_sockets=127.4.159.131
--webserver_interface=127.4.159.131
--webserver_port=0
--tserver_master_addrs=127.4.159.190:44203
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.131.keytab
--principal=kudu/127.4.159.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:39301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:47:35.325237 5315 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:47:35.325410 5315 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:47:35.325431 5315 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:47:35.326538 5315 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:47:35.326575 5315 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:47:35.326604 5315 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.159.131
I20260807 08:47:35.328004 5315 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:39301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786092453486433-4734-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786092453486433-4734-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.131.keytab
--principal=kudu/127.4.159.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.159.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786092453486433-4734-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.159.131
--webserver_port=0
--tserver_master_addrs=127.4.159.190:44203
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5315
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.159.131
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786092453486433-4734-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:47:35.328198 5315 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:47:35.328387 5315 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:47:35.332965 5326 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:47:35.332983 5315 server_base.cc:1034] running on GCE node
W20260807 08:47:35.332916 5327 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:47:35.332922 5329 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:47:35.333256 5315 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:47:35.333467 5315 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:47:35.334599 5315 hybrid_clock.cc:648] HybridClock initialized: now 1786092455334583 us; error 27 us; skew 500 ppm
Aug 07 08:47:35 dist-test-slave-6zbq krb5kdc[4743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092455, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:47:35.336769 5315 init.cc:377] Logged in from keytab as kudu/127.4.159.131@KRBTEST.COM (short username kudu)
I20260807 08:47:35.337149 5315 webserver.cc:460] Webserver started at http://127.4.159.131:37439/ using document root <none> and password file <none>
I20260807 08:47:35.337332 5315 fs_manager.cc:359] Metadata directory not provided
I20260807 08:47:35.337378 5315 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:47:35.337498 5315 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:47:35.338557 5315 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786092453486433-4734-0/minicluster-data/ts-2/data/instance:
uuid: "07af99f215ad4179bdcfbd5615e25c05"
format_stamp: "Formatted at 2026-08-07 08:47:35 on dist-test-slave-6zbq"
server_key: "c41f8116266319ea0064c4b50da8b4e4"
server_key_iv: "8faf2e7a50cb6442f4aea82f71ed16ff"
server_key_version: "encryptionkey@0"
I20260807 08:47:35.338881 5315 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786092453486433-4734-0/minicluster-data/ts-2/wal/instance:
uuid: "07af99f215ad4179bdcfbd5615e25c05"
format_stamp: "Formatted at 2026-08-07 08:47:35 on dist-test-slave-6zbq"
server_key: "c41f8116266319ea0064c4b50da8b4e4"
server_key_iv: "8faf2e7a50cb6442f4aea82f71ed16ff"
server_key_version: "encryptionkey@0"
I20260807 08:47:35.340090 5315 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:47:35.340688 5343 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:35.340840 5315 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 08:47:35.340921 5315 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786092453486433-4734-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786092453486433-4734-0/minicluster-data/ts-2/wal
uuid: "07af99f215ad4179bdcfbd5615e25c05"
format_stamp: "Formatted at 2026-08-07 08:47:35 on dist-test-slave-6zbq"
server_key: "c41f8116266319ea0064c4b50da8b4e4"
server_key_iv: "8faf2e7a50cb6442f4aea82f71ed16ff"
server_key_version: "encryptionkey@0"
I20260807 08:47:35.340981 5315 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786092453486433-4734-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786092453486433-4734-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786092453486433-4734-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:47:35.371882 5315 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:47:35.373775 5315 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:47:35.373924 5315 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:47:35.374294 5315 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:47:35.374635 5315 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:47:35.374671 5315 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:35.374699 5315 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:47:35.374722 5315 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:35.380623 5315 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.131:36843
I20260807 08:47:35.380678 5522 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.131:36843 every 8 connection(s)
I20260807 08:47:35.380990 5315 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786092453486433-4734-0/minicluster-data/ts-2/data/info.pb
Aug 07 08:47:35 dist-test-slave-6zbq krb5kdc[4743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092455, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.131@KRBTEST.COM for kudu/127.4.159.190@KRBTEST.COM
I20260807 08:47:35.389447 4864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:35.381738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.131:45405 (local address 127.4.159.190:44203)
0807 08:47:35.381795 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:35.381797 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:35.382005 (+ 208us) server_negotiation.cc:407] Connection header received
0807 08:47:35.382509 (+ 504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:35.382510 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:35.382519 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:35.382574 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:47:35.383475 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:35.383965 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:35.384893 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:35.385065 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:35.385107 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:35.387003 (+ 1896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:47:35.387004 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:47:35.387005 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:47:35.387012 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:47:35.388035 (+ 1023us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:35.388397 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:35.388398 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:35.388399 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:35.388462 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:35.388752 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:35.388753 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:35.388754 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:35.388893 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:47:35.388964 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:35.389274 (+ 310us) server_negotiation.cc:299] Negotiation successful
0807 08:47:35.389357 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260807 08:47:35.389618 5526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:35.381725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:44203 (local address 127.4.159.131:45405)
0807 08:47:35.381887 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0807 08:47:35.381928 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 08:47:35.382430 (+ 502us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:47:35.382631 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:47:35.382634 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:47:35.383204 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:47:35.383395 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:35.383400 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:35.384039 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:35.384039 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:35.384831 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:35.384833 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:35.385116 (+ 283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:35.385117 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:35.385211 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:35.385579 (+ 368us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:47:35.385588 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:47:35.386916 (+ 1328us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:47:35.388105 (+ 1189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:47:35.388111 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:47:35.388113 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:47:35.388332 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:47:35.388521 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:47:35.388522 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:47:35.388522 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:47:35.388676 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:47:35.388965 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:47:35.388971 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:47:35.389124 (+ 153us) client_negotiation.cc:769] Sending connection context
0807 08:47:35.389232 (+ 108us) client_negotiation.cc:240] Negotiation successful
0807 08:47:35.389397 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":69,"threads_started":1}
I20260807 08:47:35.390262 5523 heartbeater.cc:347] Connected to a master server at 127.4.159.190:44203
I20260807 08:47:35.390373 5523 heartbeater.cc:464] Registering TS with master...
I20260807 08:47:35.390615 5523 heartbeater.cc:511] Master 127.4.159.190:44203 requested a full tablet report, sending...
I20260807 08:47:35.390717 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 5315
I20260807 08:47:35.390799 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786092453486433-4734-0/minicluster-data/ts-2/wal/instance
I20260807 08:47:35.390964 4734 external_mini_cluster.cc:1428] Setting key ee35ab3c0c4933c02a4eee9f27829ece
I20260807 08:47:35.391067 4810 ts_manager.cc:194] Registered new tserver with Master: 07af99f215ad4179bdcfbd5615e25c05 (127.4.159.131:36843)
I20260807 08:47:35.391510 4810 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.159.131@KRBTEST.COM'} at 127.4.159.131:45405
I20260807 08:47:35.392262 4734 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 08:47:35 dist-test-slave-6zbq krb5kdc[4743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092455, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 08:47:35 dist-test-slave-6zbq krb5kdc[4743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092455, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.4.159.129@KRBTEST.COM
I20260807 08:47:35.414513 5534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:35.407575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55350 (local address 127.4.159.129:45251)
0807 08:47:35.407748 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:35.407754 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:35.407762 (+ 8us) server_negotiation.cc:407] Connection header received
0807 08:47:35.407785 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:35.407787 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:35.407800 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:35.407862 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:47:35.408381 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:35.409075 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:35.409774 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:35.410082 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:35.410142 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:35.412114 (+ 1972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:47:35.412122 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:47:35.412122 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:47:35.412128 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:47:35.413460 (+ 1332us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:35.413774 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:35.413776 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:35.413777 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:35.413830 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:35.413991 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:35.413992 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:35.413992 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:35.414116 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:47:35.414258 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:35.414305 (+ 47us) server_negotiation.cc:299] Negotiation successful
0807 08:47:35.414410 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":83,"threads_started":1}
W20260807 08:47:35.414948 4968 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55350
Aug 07 08:47:35 dist-test-slave-6zbq krb5kdc[4743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092455, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 08:47:35 dist-test-slave-6zbq krb5kdc[4743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092455, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.159.129@KRBTEST.COM
I20260807 08:47:35.434463 5534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:35.428664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55360 (local address 127.4.159.129:45251)
0807 08:47:35.428709 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:35.428711 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:35.428765 (+ 54us) server_negotiation.cc:407] Connection header received
0807 08:47:35.428828 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:35.428829 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:35.428839 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:35.428882 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:47:35.429371 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:35.429819 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:35.430535 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:35.430733 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:35.430784 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:35.432556 (+ 1772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:47:35.432558 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:47:35.432558 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:47:35.432565 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:47:35.433547 (+ 982us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:35.433823 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:35.433824 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:35.433824 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:35.433869 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:35.434021 (+ 152us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:35.434022 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:35.434023 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:35.434103 (+ 80us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:47:35.434198 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:35.434295 (+ 97us) server_negotiation.cc:299] Negotiation successful
0807 08:47:35.434381 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260807 08:47:35.434863 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 4905
I20260807 08:47:35.446100 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 5100
I20260807 08:47:35.459538 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 5315
I20260807 08:47:35.466452 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 4765
2026-08-07T08:47:35Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (1980 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786092453486433-4734-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:47:35 dist-test-slave-6zbq krb5kdc[5544](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:47:35 dist-test-slave-6zbq krb5kdc[5544](info): set up 2 sockets
Aug 07 08:47:35 dist-test-slave-6zbq krb5kdc[5544](info): commencing operation
krb5kdc: starting...
W20260807 08:47:36.575016 4734 mini_kdc.cc:121] Time spent starting KDC: real 1.082s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786092453486433-4734-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786092453486433-4734-0/krb5kdc/test-user.keytab.
Aug 07 08:47:36 dist-test-slave-6zbq krb5kdc[5544](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092456, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:47:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:47:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.159.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.159.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.159.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab.
Entry for principal kudu/127.4.159.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab.
I20260807 08:47:36.680199 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786092453486433-4734-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786092453486433-4734-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786092453486433-4734-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.159.190:36157
--webserver_interface=127.4.159.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab
--principal=kudu/127.4.159.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:36707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.159.190:36157
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:47:36.758503 5563 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:47:36.758673 5563 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:47:36.758692 5563 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:47:36.759775 5563 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:47:36.759814 5563 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:47:36.759827 5563 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:47:36.759840 5563 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:47:36.759864 5563 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:47:36.761265 5563 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:36707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786092453486433-4734-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786092453486433-4734-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.159.190:36157
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab
--principal=kudu/127.4.159.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.159.190:36157
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.159.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5563
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786092453486433-4734-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:47:36.761473 5563 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:47:36.761660 5563 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:47:36.766012 5572 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:47:36.766011 5570 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:47:36.766055 5568 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:47:36.766145 5563 server_base.cc:1034] running on GCE node
I20260807 08:47:36.766484 5563 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:47:36.766734 5563 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:47:36.767858 5563 hybrid_clock.cc:648] HybridClock initialized: now 1786092456767843 us; error 29 us; skew 500 ppm
Aug 07 08:47:36 dist-test-slave-6zbq krb5kdc[5544](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092456, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:47:36.770017 5563 init.cc:377] Logged in from keytab as kudu/127.4.159.190@KRBTEST.COM (short username kudu)
I20260807 08:47:36.770406 5563 webserver.cc:460] Webserver started at http://127.4.159.190:42993/ using document root <none> and password file <none>
I20260807 08:47:36.770573 5563 fs_manager.cc:359] Metadata directory not provided
I20260807 08:47:36.770610 5563 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:47:36.770696 5563 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:47:36.771590 5563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786092453486433-4734-0/minicluster-data/master-0/data/instance:
uuid: "64a5823cf7b9491cbf6286897bd111b2"
format_stamp: "Formatted at 2026-08-07 08:47:36 on dist-test-slave-6zbq"
server_key: "9a713b07f3805508db17b4d0a06ed98d"
server_key_iv: "0322b85db340512b670291bf71b5d7be"
server_key_version: "encryptionkey@0"
I20260807 08:47:36.771878 5563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786092453486433-4734-0/minicluster-data/master-0/wal/instance:
uuid: "64a5823cf7b9491cbf6286897bd111b2"
format_stamp: "Formatted at 2026-08-07 08:47:36 on dist-test-slave-6zbq"
server_key: "9a713b07f3805508db17b4d0a06ed98d"
server_key_iv: "0322b85db340512b670291bf71b5d7be"
server_key_version: "encryptionkey@0"
I20260807 08:47:36.772979 5563 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260807 08:47:36.773615 5582 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:36.773767 5563 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 08:47:36.773859 5563 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786092453486433-4734-0/minicluster-data/master-0/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786092453486433-4734-0/minicluster-data/master-0/wal
uuid: "64a5823cf7b9491cbf6286897bd111b2"
format_stamp: "Formatted at 2026-08-07 08:47:36 on dist-test-slave-6zbq"
server_key: "9a713b07f3805508db17b4d0a06ed98d"
server_key_iv: "0322b85db340512b670291bf71b5d7be"
server_key_version: "encryptionkey@0"
I20260807 08:47:36.773918 5563 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786092453486433-4734-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786092453486433-4734-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786092453486433-4734-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:47:36.786877 5563 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:47:36.788357 5563 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:47:36.788475 5563 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:47:36.792040 5563 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.190:36157
I20260807 08:47:36.792078 5655 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.190:36157 every 8 connection(s)
I20260807 08:47:36.792351 5563 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
I20260807 08:47:36.793084 5656 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:47:36.793915 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 5563
I20260807 08:47:36.793989 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786092453486433-4734-0/minicluster-data/master-0/wal/instance
I20260807 08:47:36.794135 4734 external_mini_cluster.cc:1428] Setting key b05b112dd9aa7f22f13d9efa8a44f3a7
I20260807 08:47:36.795615 5656 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64a5823cf7b9491cbf6286897bd111b2: Bootstrap starting.
I20260807 08:47:36.796221 5656 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 64a5823cf7b9491cbf6286897bd111b2: Neither blocks nor log segments found. Creating new log.
I20260807 08:47:36.796536 5656 log.cc:824] T 00000000000000000000000000000000 P 64a5823cf7b9491cbf6286897bd111b2: Log is configured to *not* fsync() on all Append() calls
I20260807 08:47:36.797204 5656 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64a5823cf7b9491cbf6286897bd111b2: No bootstrap required, opened a new log
I20260807 08:47:36.799377 5656 raft_consensus.cc:348] T 00000000000000000000000000000000 P 64a5823cf7b9491cbf6286897bd111b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64a5823cf7b9491cbf6286897bd111b2" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 36157 } }
I20260807 08:47:36.799495 5656 raft_consensus.cc:374] T 00000000000000000000000000000000 P 64a5823cf7b9491cbf6286897bd111b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:47:36.799517 5656 raft_consensus.cc:729] T 00000000000000000000000000000000 P 64a5823cf7b9491cbf6286897bd111b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64a5823cf7b9491cbf6286897bd111b2, State: Initialized, Role: FOLLOWER
I20260807 08:47:36.799618 5656 consensus_queue.cc:260] T 00000000000000000000000000000000 P 64a5823cf7b9491cbf6286897bd111b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64a5823cf7b9491cbf6286897bd111b2" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 36157 } }
I20260807 08:47:36.799690 5656 raft_consensus.cc:388] T 00000000000000000000000000000000 P 64a5823cf7b9491cbf6286897bd111b2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:47:36.799716 5656 raft_consensus.cc:482] T 00000000000000000000000000000000 P 64a5823cf7b9491cbf6286897bd111b2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:47:36.799760 5656 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 64a5823cf7b9491cbf6286897bd111b2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:47:36.800238 5656 raft_consensus.cc:504] T 00000000000000000000000000000000 P 64a5823cf7b9491cbf6286897bd111b2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64a5823cf7b9491cbf6286897bd111b2" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 36157 } }
I20260807 08:47:36.800323 5656 leader_election.cc:302] T 00000000000000000000000000000000 P 64a5823cf7b9491cbf6286897bd111b2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 64a5823cf7b9491cbf6286897bd111b2; no voters:
Aug 07 08:47:36 dist-test-slave-6zbq krb5kdc[5544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092456, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.159.190@KRBTEST.COM
I20260807 08:47:36.800484 5656 leader_election.cc:288] T 00000000000000000000000000000000 P 64a5823cf7b9491cbf6286897bd111b2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:47:36.800529 5664 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 64a5823cf7b9491cbf6286897bd111b2 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:47:36.800637 5664 raft_consensus.cc:686] T 00000000000000000000000000000000 P 64a5823cf7b9491cbf6286897bd111b2 [term 1 LEADER]: Becoming Leader. State: Replica: 64a5823cf7b9491cbf6286897bd111b2, State: Running, Role: LEADER
I20260807 08:47:36.800721 5656 sys_catalog.cc:564] T 00000000000000000000000000000000 P 64a5823cf7b9491cbf6286897bd111b2 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:47:36.800724 5664 consensus_queue.cc:237] T 00000000000000000000000000000000 P 64a5823cf7b9491cbf6286897bd111b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64a5823cf7b9491cbf6286897bd111b2" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 36157 } }
I20260807 08:47:36.801146 5665 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64a5823cf7b9491cbf6286897bd111b2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 64a5823cf7b9491cbf6286897bd111b2. Latest consensus state: current_term: 1 leader_uuid: "64a5823cf7b9491cbf6286897bd111b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64a5823cf7b9491cbf6286897bd111b2" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 36157 } } }
I20260807 08:47:36.801229 5665 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64a5823cf7b9491cbf6286897bd111b2 [sys.catalog]: This master's current role is: LEADER
I20260807 08:47:36.801349 5666 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64a5823cf7b9491cbf6286897bd111b2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "64a5823cf7b9491cbf6286897bd111b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64a5823cf7b9491cbf6286897bd111b2" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 36157 } } }
I20260807 08:47:36.801446 5679 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:47:36.801918 5679 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:47:36.801460 5666 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64a5823cf7b9491cbf6286897bd111b2 [sys.catalog]: This master's current role is: LEADER
I20260807 08:47:36.803542 5679 catalog_manager.cc:1347] Generated new cluster ID: 7ce0b35ed2274b6e8705f1e143d9eaa5
I20260807 08:47:36.803603 5679 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:47:36.804884 5662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:36.794626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44930 (local address 127.4.159.190:36157)
0807 08:47:36.794804 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:36.794812 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:36.794847 (+ 35us) server_negotiation.cc:407] Connection header received
0807 08:47:36.795312 (+ 465us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:36.795320 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:36.795379 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:36.795522 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:47:36.796048 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:36.797296 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:36.798003 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:36.798617 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:36.798750 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:36.800988 (+ 2238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:47:36.800992 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:47:36.800992 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:47:36.801003 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:47:36.803303 (+ 2300us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:36.803706 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:36.803709 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:36.803710 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:36.803818 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:36.804030 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:36.804031 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:36.804031 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:36.804320 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:47:36.804470 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:36.804543 (+ 73us) server_negotiation.cc:299] Negotiation successful
0807 08:47:36.804671 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":130,"thread_start_us":73,"threads_started":1}
I20260807 08:47:36.817167 5679 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:47:36.817817 5679 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:47:36.826902 5679 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 64a5823cf7b9491cbf6286897bd111b2: Generated new TSK 0
I20260807 08:47:36.827051 5679 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.159.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.159.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.159.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.129.keytab.
Entry for principal kudu/127.4.159.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.129.keytab.
I20260807 08:47:36.853600 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786092453486433-4734-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786092453486433-4734-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786092453486433-4734-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786092453486433-4734-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.159.129:0
--local_ip_for_outbound_sockets=127.4.159.129
--webserver_interface=127.4.159.129
--webserver_port=0
--tserver_master_addrs=127.4.159.190:36157
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.129.keytab
--principal=kudu/127.4.159.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:36707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:47:36.931695 5698 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:47:36.931885 5698 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:47:36.931907 5698 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:47:36.933009 5698 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:47:36.933048 5698 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:47:36.933073 5698 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.159.129
I20260807 08:47:36.934535 5698 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:36707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786092453486433-4734-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786092453486433-4734-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.129.keytab
--principal=kudu/127.4.159.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.159.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786092453486433-4734-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.159.129
--webserver_port=0
--tserver_master_addrs=127.4.159.190:36157
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5698
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.159.129
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786092453486433-4734-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:47:36.934738 5698 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:47:36.934942 5698 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:47:36.939535 5706 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:47:36.939548 5703 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:47:36.939563 5704 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:47:36.939707 5698 server_base.cc:1034] running on GCE node
I20260807 08:47:36.939908 5698 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:47:36.940114 5698 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:47:36.941246 5698 hybrid_clock.cc:648] HybridClock initialized: now 1786092456941234 us; error 28 us; skew 500 ppm
Aug 07 08:47:36 dist-test-slave-6zbq krb5kdc[5544](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092456, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:47:36.943524 5698 init.cc:377] Logged in from keytab as kudu/127.4.159.129@KRBTEST.COM (short username kudu)
I20260807 08:47:36.943908 5698 webserver.cc:460] Webserver started at http://127.4.159.129:36383/ using document root <none> and password file <none>
I20260807 08:47:36.944085 5698 fs_manager.cc:359] Metadata directory not provided
I20260807 08:47:36.944129 5698 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:47:36.944252 5698 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:47:36.945171 5698 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786092453486433-4734-0/minicluster-data/ts-0/data/instance:
uuid: "0234bcd3c17148fe8b56675f98cd2c92"
format_stamp: "Formatted at 2026-08-07 08:47:36 on dist-test-slave-6zbq"
server_key: "3392f2579844d03f1f9054ea95e4ecb6"
server_key_iv: "0b03f6a5505bee3e576bf403349ef625"
server_key_version: "encryptionkey@0"
I20260807 08:47:36.945482 5698 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786092453486433-4734-0/minicluster-data/ts-0/wal/instance:
uuid: "0234bcd3c17148fe8b56675f98cd2c92"
format_stamp: "Formatted at 2026-08-07 08:47:36 on dist-test-slave-6zbq"
server_key: "3392f2579844d03f1f9054ea95e4ecb6"
server_key_iv: "0b03f6a5505bee3e576bf403349ef625"
server_key_version: "encryptionkey@0"
I20260807 08:47:36.946655 5698 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 08:47:36.947261 5715 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:36.947403 5698 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 08:47:36.947484 5698 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786092453486433-4734-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786092453486433-4734-0/minicluster-data/ts-0/wal
uuid: "0234bcd3c17148fe8b56675f98cd2c92"
format_stamp: "Formatted at 2026-08-07 08:47:36 on dist-test-slave-6zbq"
server_key: "3392f2579844d03f1f9054ea95e4ecb6"
server_key_iv: "0b03f6a5505bee3e576bf403349ef625"
server_key_version: "encryptionkey@0"
I20260807 08:47:36.947540 5698 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786092453486433-4734-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786092453486433-4734-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786092453486433-4734-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:47:36.957837 5698 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:47:36.959371 5698 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:47:36.959492 5698 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:47:36.959671 5698 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:47:36.959923 5698 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:47:36.959954 5698 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:36.959978 5698 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:47:36.959995 5698 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:36.965658 5698 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.129:41481
I20260807 08:47:36.965713 5878 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.129:41481 every 8 connection(s)
I20260807 08:47:36.965962 5698 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786092453486433-4734-0/minicluster-data/ts-0/data/info.pb
I20260807 08:47:36.967706 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 5698
I20260807 08:47:36.967803 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786092453486433-4734-0/minicluster-data/ts-0/wal/instance
I20260807 08:47:36.967978 4734 external_mini_cluster.cc:1428] Setting key 19b8d87db26efa1535ba7ec0bfcec69c
Aug 07 08:47:36 dist-test-slave-6zbq krb5kdc[5544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092456, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.129@KRBTEST.COM for kudu/127.4.159.190@KRBTEST.COM
I20260807 08:47:36.975411 5662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:36.966596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.129:46355 (local address 127.4.159.190:36157)
0807 08:47:36.966667 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:36.966669 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:36.966805 (+ 136us) server_negotiation.cc:407] Connection header received
0807 08:47:36.967774 (+ 969us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:36.967775 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:36.967788 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:36.967981 (+ 193us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:47:36.968601 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:36.969093 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:36.970413 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:36.970699 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:36.970778 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:36.972888 (+ 2110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:47:36.972889 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:47:36.972890 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:47:36.972896 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:47:36.974066 (+ 1170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:36.974562 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:36.974562 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:36.974563 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:36.974609 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:36.974870 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:36.974870 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:36.974870 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:36.974962 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:47:36.975005 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:36.975290 (+ 285us) server_negotiation.cc:299] Negotiation successful
0807 08:47:36.975351 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 08:47:36.975651 5882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:36.966567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:36157 (local address 127.4.159.129:46355)
0807 08:47:36.966702 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0807 08:47:36.966734 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0807 08:47:36.967673 (+ 939us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:47:36.968073 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:47:36.968078 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:47:36.968350 (+ 272us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:47:36.968522 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:36.968526 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:36.969182 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:36.969183 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:36.970298 (+ 1115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:36.970301 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:36.970782 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:36.970783 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:36.970883 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:36.971289 (+ 406us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:47:36.971297 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:47:36.972792 (+ 1495us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:47:36.974143 (+ 1351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:47:36.974152 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:47:36.974154 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:47:36.974494 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:47:36.974666 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:47:36.974666 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:47:36.974667 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:47:36.974805 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:47:36.975012 (+ 207us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:47:36.975017 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:47:36.975171 (+ 154us) client_negotiation.cc:769] Sending connection context
0807 08:47:36.975284 (+ 113us) client_negotiation.cc:240] Negotiation successful
0807 08:47:36.975441 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":76,"threads_started":1}
I20260807 08:47:36.976130 5880 heartbeater.cc:347] Connected to a master server at 127.4.159.190:36157
I20260807 08:47:36.976210 5880 heartbeater.cc:464] Registering TS with master...
I20260807 08:47:36.976369 5880 heartbeater.cc:511] Master 127.4.159.190:36157 requested a full tablet report, sending...
I20260807 08:47:36.976861 5608 ts_manager.cc:194] Registered new tserver with Master: 0234bcd3c17148fe8b56675f98cd2c92 (127.4.159.129:41481)
WARNING: no policy specified for kudu/127.4.159.130@KRBTEST.COM; defaulting to no policy
I20260807 08:47:36.977487 5608 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.159.129@KRBTEST.COM'} at 127.4.159.129:46355
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.159.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.159.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.130.keytab.
Entry for principal kudu/127.4.159.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.130.keytab.
I20260807 08:47:36.990682 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786092453486433-4734-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786092453486433-4734-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786092453486433-4734-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786092453486433-4734-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.159.130:0
--local_ip_for_outbound_sockets=127.4.159.130
--webserver_interface=127.4.159.130
--webserver_port=0
--tserver_master_addrs=127.4.159.190:36157
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.130.keytab
--principal=kudu/127.4.159.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:36707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:47:37.067781 5888 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:47:37.067953 5888 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:47:37.067984 5888 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:47:37.069133 5888 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:47:37.069178 5888 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:47:37.069216 5888 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.159.130
I20260807 08:47:37.070676 5888 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:36707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786092453486433-4734-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786092453486433-4734-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.130.keytab
--principal=kudu/127.4.159.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.159.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786092453486433-4734-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.159.130
--webserver_port=0
--tserver_master_addrs=127.4.159.190:36157
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5888
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.159.130
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786092453486433-4734-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:47:37.070894 5888 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:47:37.071086 5888 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:47:37.075592 5899 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:47:37.075711 5888 server_base.cc:1034] running on GCE node
W20260807 08:47:37.075594 5895 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:47:37.075594 5896 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:47:37.076041 5888 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:47:37.076282 5888 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:47:37.077414 5888 hybrid_clock.cc:648] HybridClock initialized: now 1786092457077401 us; error 28 us; skew 500 ppm
Aug 07 08:47:37 dist-test-slave-6zbq krb5kdc[5544](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092457, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:47:37.079658 5888 init.cc:377] Logged in from keytab as kudu/127.4.159.130@KRBTEST.COM (short username kudu)
I20260807 08:47:37.080034 5888 webserver.cc:460] Webserver started at http://127.4.159.130:34629/ using document root <none> and password file <none>
I20260807 08:47:37.080237 5888 fs_manager.cc:359] Metadata directory not provided
I20260807 08:47:37.080286 5888 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:47:37.080403 5888 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:47:37.081357 5888 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786092453486433-4734-0/minicluster-data/ts-1/data/instance:
uuid: "734acbba8fe5459db24d0b3422150dc3"
format_stamp: "Formatted at 2026-08-07 08:47:37 on dist-test-slave-6zbq"
server_key: "a710a73a74e867dfac31a141796b5a92"
server_key_iv: "5409e3e4b9b5ce3696a87bd13823c490"
server_key_version: "encryptionkey@0"
I20260807 08:47:37.081660 5888 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786092453486433-4734-0/minicluster-data/ts-1/wal/instance:
uuid: "734acbba8fe5459db24d0b3422150dc3"
format_stamp: "Formatted at 2026-08-07 08:47:37 on dist-test-slave-6zbq"
server_key: "a710a73a74e867dfac31a141796b5a92"
server_key_iv: "5409e3e4b9b5ce3696a87bd13823c490"
server_key_version: "encryptionkey@0"
I20260807 08:47:37.082938 5888 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 08:47:37.083576 5906 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:37.083715 5888 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 08:47:37.083796 5888 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786092453486433-4734-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786092453486433-4734-0/minicluster-data/ts-1/wal
uuid: "734acbba8fe5459db24d0b3422150dc3"
format_stamp: "Formatted at 2026-08-07 08:47:37 on dist-test-slave-6zbq"
server_key: "a710a73a74e867dfac31a141796b5a92"
server_key_iv: "5409e3e4b9b5ce3696a87bd13823c490"
server_key_version: "encryptionkey@0"
I20260807 08:47:37.083853 5888 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786092453486433-4734-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786092453486433-4734-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786092453486433-4734-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:47:37.093806 5888 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:47:37.095402 5888 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:47:37.095521 5888 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:47:37.095697 5888 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:47:37.095950 5888 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:47:37.095983 5888 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:37.096007 5888 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:47:37.096035 5888 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:37.101480 5888 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.130:39613
I20260807 08:47:37.101521 6071 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.130:39613 every 8 connection(s)
I20260807 08:47:37.101786 5888 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786092453486433-4734-0/minicluster-data/ts-1/data/info.pb
I20260807 08:47:37.104526 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 5888
I20260807 08:47:37.104614 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786092453486433-4734-0/minicluster-data/ts-1/wal/instance
I20260807 08:47:37.104773 4734 external_mini_cluster.cc:1428] Setting key 8d3a8d105ec24df5861b8b6b534170b8
Aug 07 08:47:37 dist-test-slave-6zbq krb5kdc[5544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092457, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.130@KRBTEST.COM for kudu/127.4.159.190@KRBTEST.COM
I20260807 08:47:37.110795 5662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:37.102451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.130:37633 (local address 127.4.159.190:36157)
0807 08:47:37.102513 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:37.102515 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:37.102644 (+ 129us) server_negotiation.cc:407] Connection header received
0807 08:47:37.103457 (+ 813us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:37.103458 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:37.103468 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:37.103518 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:47:37.104033 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:37.104515 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:37.105652 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:37.105845 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:37.105899 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:37.108271 (+ 2372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:47:37.108273 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:47:37.108273 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:47:37.108280 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:47:37.109457 (+ 1177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:37.109912 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:37.109912 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:37.109913 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:37.109963 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:37.110200 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:37.110200 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:37.110201 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:37.110294 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:47:37.110364 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:37.110666 (+ 302us) server_negotiation.cc:299] Negotiation successful
0807 08:47:37.110732 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260807 08:47:37.111018 6075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:37.102398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:36157 (local address 127.4.159.130:37633)
0807 08:47:37.102538 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0807 08:47:37.102571 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0807 08:47:37.103371 (+ 800us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:47:37.103566 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:47:37.103573 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:47:37.103802 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:47:37.103972 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:37.103976 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:37.104596 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:37.104597 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:37.105570 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:37.105574 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:37.105924 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:37.105925 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:37.106065 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:37.106537 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:47:37.106547 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:47:37.108184 (+ 1637us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:47:37.109544 (+ 1360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:47:37.109552 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:47:37.109555 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:47:37.109845 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:47:37.110016 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:47:37.110016 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:47:37.110017 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:47:37.110145 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:47:37.110367 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:47:37.110373 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:47:37.110530 (+ 157us) client_negotiation.cc:769] Sending connection context
0807 08:47:37.110654 (+ 124us) client_negotiation.cc:240] Negotiation successful
0807 08:47:37.110810 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":73,"threads_started":1}
I20260807 08:47:37.111469 6072 heartbeater.cc:347] Connected to a master server at 127.4.159.190:36157
I20260807 08:47:37.111552 6072 heartbeater.cc:464] Registering TS with master...
I20260807 08:47:37.111706 6072 heartbeater.cc:511] Master 127.4.159.190:36157 requested a full tablet report, sending...
I20260807 08:47:37.112021 5608 ts_manager.cc:194] Registered new tserver with Master: 734acbba8fe5459db24d0b3422150dc3 (127.4.159.130:39613)
I20260807 08:47:37.112365 5608 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.159.130@KRBTEST.COM'} at 127.4.159.130:37633
WARNING: no policy specified for kudu/127.4.159.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.159.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.159.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.131.keytab.
Entry for principal kudu/127.4.159.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.131.keytab.
I20260807 08:47:37.128340 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786092453486433-4734-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786092453486433-4734-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786092453486433-4734-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786092453486433-4734-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.159.131:0
--local_ip_for_outbound_sockets=127.4.159.131
--webserver_interface=127.4.159.131
--webserver_port=0
--tserver_master_addrs=127.4.159.190:36157
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.131.keytab
--principal=kudu/127.4.159.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:36707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:47:37.205816 6078 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:47:37.205987 6078 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:47:37.206009 6078 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:47:37.207073 6078 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:47:37.207106 6078 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:47:37.207131 6078 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.159.131
I20260807 08:47:37.208487 6078 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:36707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786092453486433-4734-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786092453486433-4734-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.131.keytab
--principal=kudu/127.4.159.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.159.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786092453486433-4734-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.159.131
--webserver_port=0
--tserver_master_addrs=127.4.159.190:36157
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6078
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.159.131
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786092453486433-4734-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:47:37.208669 6078 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:47:37.208841 6078 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:47:37.213434 6084 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:47:37.213454 6085 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:47:37.213565 6078 server_base.cc:1034] running on GCE node
W20260807 08:47:37.213693 6087 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:47:37.213896 6078 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:47:37.214126 6078 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:47:37.215268 6078 hybrid_clock.cc:648] HybridClock initialized: now 1786092457215256 us; error 29 us; skew 500 ppm
Aug 07 08:47:37 dist-test-slave-6zbq krb5kdc[5544](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092457, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:47:37.217553 6078 init.cc:377] Logged in from keytab as kudu/127.4.159.131@KRBTEST.COM (short username kudu)
I20260807 08:47:37.217952 6078 webserver.cc:460] Webserver started at http://127.4.159.131:41081/ using document root <none> and password file <none>
I20260807 08:47:37.218132 6078 fs_manager.cc:359] Metadata directory not provided
I20260807 08:47:37.218178 6078 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:47:37.218294 6078 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:47:37.219239 6078 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786092453486433-4734-0/minicluster-data/ts-2/data/instance:
uuid: "9dadd97d8b3a4b01a5e53105e077f90a"
format_stamp: "Formatted at 2026-08-07 08:47:37 on dist-test-slave-6zbq"
server_key: "847abf8be75d57955b6dabbea43d43f2"
server_key_iv: "1d9f6981c36a09f0679a16db7f7a6acd"
server_key_version: "encryptionkey@0"
I20260807 08:47:37.219552 6078 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786092453486433-4734-0/minicluster-data/ts-2/wal/instance:
uuid: "9dadd97d8b3a4b01a5e53105e077f90a"
format_stamp: "Formatted at 2026-08-07 08:47:37 on dist-test-slave-6zbq"
server_key: "847abf8be75d57955b6dabbea43d43f2"
server_key_iv: "1d9f6981c36a09f0679a16db7f7a6acd"
server_key_version: "encryptionkey@0"
I20260807 08:47:37.220738 6078 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 08:47:37.221352 6098 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:37.221510 6078 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 08:47:37.221588 6078 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786092453486433-4734-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786092453486433-4734-0/minicluster-data/ts-2/wal
uuid: "9dadd97d8b3a4b01a5e53105e077f90a"
format_stamp: "Formatted at 2026-08-07 08:47:37 on dist-test-slave-6zbq"
server_key: "847abf8be75d57955b6dabbea43d43f2"
server_key_iv: "1d9f6981c36a09f0679a16db7f7a6acd"
server_key_version: "encryptionkey@0"
I20260807 08:47:37.221647 6078 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786092453486433-4734-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786092453486433-4734-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786092453486433-4734-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:47:37.241933 6078 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:47:37.243534 6078 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:47:37.243662 6078 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:47:37.243914 6078 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:47:37.244184 6078 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:47:37.244217 6078 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:37.244244 6078 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:47:37.244261 6078 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:37.249619 6078 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.131:35331
I20260807 08:47:37.249663 6266 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.131:35331 every 8 connection(s)
I20260807 08:47:37.249925 6078 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786092453486433-4734-0/minicluster-data/ts-2/data/info.pb
I20260807 08:47:37.252254 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 6078
I20260807 08:47:37.252358 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786092453486433-4734-0/minicluster-data/ts-2/wal/instance
I20260807 08:47:37.252516 4734 external_mini_cluster.cc:1428] Setting key ae5095a1cd777dbf714781948e1769d8
Aug 07 08:47:37 dist-test-slave-6zbq krb5kdc[5544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092457, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.131@KRBTEST.COM for kudu/127.4.159.190@KRBTEST.COM
I20260807 08:47:37.258231 5662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:37.250629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.131:49843 (local address 127.4.159.190:36157)
0807 08:47:37.250681 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:37.250683 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:37.250839 (+ 156us) server_negotiation.cc:407] Connection header received
0807 08:47:37.251690 (+ 851us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:37.251691 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:37.251700 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:37.251752 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:47:37.252276 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:37.252995 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:37.254023 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:37.254231 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:37.254281 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:37.256056 (+ 1775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:47:37.256057 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:47:37.256057 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:47:37.256063 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:47:37.257291 (+ 1228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:37.257586 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:37.257586 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:37.257587 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:37.257631 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:37.257801 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:37.257802 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:37.257802 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:37.257893 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:47:37.257936 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:37.258116 (+ 180us) server_negotiation.cc:299] Negotiation successful
0807 08:47:37.258169 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260807 08:47:37.258405 6271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:37.250582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:36157 (local address 127.4.159.131:49843)
0807 08:47:37.250725 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0807 08:47:37.250770 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 08:47:37.251610 (+ 840us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:47:37.251802 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:47:37.251805 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:47:37.252024 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:47:37.252191 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:37.252195 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:37.253093 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:37.253094 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:37.253952 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:37.253955 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:37.254290 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:37.254290 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:37.254426 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:37.254777 (+ 351us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:47:37.254784 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:47:37.255979 (+ 1195us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:47:37.257358 (+ 1379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:47:37.257361 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:47:37.257362 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:47:37.257537 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:47:37.257672 (+ 135us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:47:37.257672 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:47:37.257673 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:47:37.257762 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:47:37.257933 (+ 171us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:47:37.257936 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:47:37.258031 (+ 95us) client_negotiation.cc:769] Sending connection context
0807 08:47:37.258101 (+ 70us) client_negotiation.cc:240] Negotiation successful
0807 08:47:37.258200 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":64,"threads_started":1}
I20260807 08:47:37.258839 6269 heartbeater.cc:347] Connected to a master server at 127.4.159.190:36157
I20260807 08:47:37.258924 6269 heartbeater.cc:464] Registering TS with master...
I20260807 08:47:37.259081 6269 heartbeater.cc:511] Master 127.4.159.190:36157 requested a full tablet report, sending...
I20260807 08:47:37.259407 5608 ts_manager.cc:194] Registered new tserver with Master: 9dadd97d8b3a4b01a5e53105e077f90a (127.4.159.131:35331)
I20260807 08:47:37.259749 5608 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.159.131@KRBTEST.COM'} at 127.4.159.131:49843
I20260807 08:47:37.263948 4734 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 08:47:37.271005 5662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:37.265853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44934 (local address 127.4.159.190:36157)
0807 08:47:37.265920 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:37.265921 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:37.265980 (+ 59us) server_negotiation.cc:407] Connection header received
0807 08:47:37.266039 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:37.266040 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:37.266050 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:37.266105 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:47:37.266647 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:37.267097 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:37.267780 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:37.267963 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:37.268014 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:37.269048 (+ 1034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:47:37.269049 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:47:37.269049 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:47:37.269056 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:47:37.270003 (+ 947us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:37.270285 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:37.270286 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:37.270286 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:37.270333 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:37.270506 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:37.270507 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:37.270507 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:37.270644 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:47:37.270730 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:37.270856 (+ 126us) server_negotiation.cc:299] Negotiation successful
0807 08:47:37.270925 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 08:47:37.273407 5608 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44934:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 08:47:37.275254 5608 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:47:37.281605 6278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:37.277631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.130:39613 (local address 127.0.0.1:43966)
0807 08:47:37.277748 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0807 08:47:37.277791 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 08:47:37.277856 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:47:37.278364 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:47:37.278367 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:47:37.278369 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:47:37.278428 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:37.278429 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:37.279639 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:37.279640 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:37.280693 (+ 1053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:37.280694 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:37.281327 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:37.281328 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:37.281410 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:37.281424 (+ 14us) client_negotiation.cc:769] Sending connection context
0807 08:47:37.281485 (+ 61us) client_negotiation.cc:240] Negotiation successful
0807 08:47:37.281535 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90,"mutex_wait_us":21,"thread_start_us":65,"threads_started":1}
I20260807 08:47:37.281826 6279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:37.277729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43966 (local address 127.4.159.130:39613)
0807 08:47:37.277951 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:37.277955 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:37.277965 (+ 10us) server_negotiation.cc:407] Connection header received
0807 08:47:37.278002 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:37.278004 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:37.278242 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:37.278333 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:47:37.278491 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:37.279546 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:37.280765 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:37.281254 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:37.281316 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:37.281371 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:37.281701 (+ 330us) server_negotiation.cc:299] Negotiation successful
0807 08:47:37.281750 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":109,"threads_started":1}
I20260807 08:47:37.281917 6281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:37.277912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39926 (local address 127.4.159.131:35331)
0807 08:47:37.278102 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:37.278105 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:37.278134 (+ 29us) server_negotiation.cc:407] Connection header received
0807 08:47:37.278244 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:37.278247 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:37.278259 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:37.278328 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:47:37.278575 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:37.279191 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:37.280631 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:37.281328 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:37.281394 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:37.281451 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:37.281813 (+ 362us) server_negotiation.cc:299] Negotiation successful
0807 08:47:37.281854 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":78,"threads_started":1}
I20260807 08:47:37.282058 6282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:37.277832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.131:35331 (local address 127.0.0.1:39926)
0807 08:47:37.278076 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0807 08:47:37.278086 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 08:47:37.278146 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:47:37.278502 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:47:37.278503 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:47:37.278504 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:47:37.278530 (+ 26us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:37.278531 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:37.279286 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:37.279286 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:37.280548 (+ 1262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:37.280550 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:37.281488 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:37.281489 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:37.281559 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:37.281563 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 08:47:37.281605 (+ 42us) client_negotiation.cc:240] Negotiation successful
0807 08:47:37.281643 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":45,"threads_started":1}
I20260807 08:47:37.282088 6280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:37.277964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.129:41481 (local address 127.0.0.1:59764)
0807 08:47:37.278134 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0807 08:47:37.278143 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 08:47:37.278180 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:47:37.278437 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:47:37.278437 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:47:37.278439 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:47:37.278476 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:37.278477 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:37.279809 (+ 1332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:37.279809 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:37.280578 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:37.280581 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:37.281674 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:37.281675 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:37.281730 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:37.281733 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 08:47:37.281774 (+ 41us) client_negotiation.cc:240] Negotiation successful
0807 08:47:37.281807 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"mutex_wait_us":1,"thread_start_us":70,"threads_started":1}
I20260807 08:47:37.282302 6283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:37.278071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59764 (local address 127.4.159.129:41481)
0807 08:47:37.278223 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:37.278227 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:37.278236 (+ 9us) server_negotiation.cc:407] Connection header received
0807 08:47:37.278273 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:37.278275 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:37.278288 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:37.278439 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:47:37.278521 (+ 82us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:37.279727 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:37.280673 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:37.281533 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:37.281590 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:37.281643 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:37.282180 (+ 537us) server_negotiation.cc:299] Negotiation successful
0807 08:47:37.282227 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":91,"threads_started":1}
I20260807 08:47:37.282598 5980 tablet_service.cc:1467] Processing CreateTablet for tablet ce694c5f05cf43dfbdc689171f28ec71 (DEFAULT_TABLE table=test-table [id=71ab00b362e746f0adbcc9003e56d08c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:47:37.282742 6169 tablet_service.cc:1467] Processing CreateTablet for tablet ce694c5f05cf43dfbdc689171f28ec71 (DEFAULT_TABLE table=test-table [id=71ab00b362e746f0adbcc9003e56d08c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:47:37.282848 5784 tablet_service.cc:1467] Processing CreateTablet for tablet ce694c5f05cf43dfbdc689171f28ec71 (DEFAULT_TABLE table=test-table [id=71ab00b362e746f0adbcc9003e56d08c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:47:37.282939 5980 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce694c5f05cf43dfbdc689171f28ec71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:47:37.282997 6169 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce694c5f05cf43dfbdc689171f28ec71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:47:37.283095 5784 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce694c5f05cf43dfbdc689171f28ec71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:47:37.285561 6284 tablet_bootstrap.cc:492] T ce694c5f05cf43dfbdc689171f28ec71 P 0234bcd3c17148fe8b56675f98cd2c92: Bootstrap starting.
I20260807 08:47:37.286089 6286 tablet_bootstrap.cc:492] T ce694c5f05cf43dfbdc689171f28ec71 P 9dadd97d8b3a4b01a5e53105e077f90a: Bootstrap starting.
I20260807 08:47:37.286533 6284 tablet_bootstrap.cc:654] T ce694c5f05cf43dfbdc689171f28ec71 P 0234bcd3c17148fe8b56675f98cd2c92: Neither blocks nor log segments found. Creating new log.
I20260807 08:47:37.286799 6286 tablet_bootstrap.cc:654] T ce694c5f05cf43dfbdc689171f28ec71 P 9dadd97d8b3a4b01a5e53105e077f90a: Neither blocks nor log segments found. Creating new log.
I20260807 08:47:37.287400 6286 log.cc:824] T ce694c5f05cf43dfbdc689171f28ec71 P 9dadd97d8b3a4b01a5e53105e077f90a: Log is configured to *not* fsync() on all Append() calls
I20260807 08:47:37.287401 6284 log.cc:824] T ce694c5f05cf43dfbdc689171f28ec71 P 0234bcd3c17148fe8b56675f98cd2c92: Log is configured to *not* fsync() on all Append() calls
I20260807 08:47:37.287737 6287 tablet_bootstrap.cc:492] T ce694c5f05cf43dfbdc689171f28ec71 P 734acbba8fe5459db24d0b3422150dc3: Bootstrap starting.
I20260807 08:47:37.288192 6286 tablet_bootstrap.cc:492] T ce694c5f05cf43dfbdc689171f28ec71 P 9dadd97d8b3a4b01a5e53105e077f90a: No bootstrap required, opened a new log
I20260807 08:47:37.288269 6286 ts_tablet_manager.cc:1397] T ce694c5f05cf43dfbdc689171f28ec71 P 9dadd97d8b3a4b01a5e53105e077f90a: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260807 08:47:37.288295 6284 tablet_bootstrap.cc:492] T ce694c5f05cf43dfbdc689171f28ec71 P 0234bcd3c17148fe8b56675f98cd2c92: No bootstrap required, opened a new log
I20260807 08:47:37.288339 6287 tablet_bootstrap.cc:654] T ce694c5f05cf43dfbdc689171f28ec71 P 734acbba8fe5459db24d0b3422150dc3: Neither blocks nor log segments found. Creating new log.
I20260807 08:47:37.288362 6284 ts_tablet_manager.cc:1397] T ce694c5f05cf43dfbdc689171f28ec71 P 0234bcd3c17148fe8b56675f98cd2c92: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 08:47:37.288911 6287 log.cc:824] T ce694c5f05cf43dfbdc689171f28ec71 P 734acbba8fe5459db24d0b3422150dc3: Log is configured to *not* fsync() on all Append() calls
I20260807 08:47:37.289789 6287 tablet_bootstrap.cc:492] T ce694c5f05cf43dfbdc689171f28ec71 P 734acbba8fe5459db24d0b3422150dc3: No bootstrap required, opened a new log
I20260807 08:47:37.289857 6287 ts_tablet_manager.cc:1397] T ce694c5f05cf43dfbdc689171f28ec71 P 734acbba8fe5459db24d0b3422150dc3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 08:47:37.289919 6284 raft_consensus.cc:348] T ce694c5f05cf43dfbdc689171f28ec71 P 0234bcd3c17148fe8b56675f98cd2c92 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "734acbba8fe5459db24d0b3422150dc3" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 39613 } } peers { permanent_uuid: "9dadd97d8b3a4b01a5e53105e077f90a" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 35331 } } peers { permanent_uuid: "0234bcd3c17148fe8b56675f98cd2c92" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 41481 } }
I20260807 08:47:37.290063 6284 raft_consensus.cc:374] T ce694c5f05cf43dfbdc689171f28ec71 P 0234bcd3c17148fe8b56675f98cd2c92 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:47:37.290102 6284 raft_consensus.cc:729] T ce694c5f05cf43dfbdc689171f28ec71 P 0234bcd3c17148fe8b56675f98cd2c92 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0234bcd3c17148fe8b56675f98cd2c92, State: Initialized, Role: FOLLOWER
I20260807 08:47:37.290227 6284 consensus_queue.cc:260] T ce694c5f05cf43dfbdc689171f28ec71 P 0234bcd3c17148fe8b56675f98cd2c92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "734acbba8fe5459db24d0b3422150dc3" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 39613 } } peers { permanent_uuid: "9dadd97d8b3a4b01a5e53105e077f90a" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 35331 } } peers { permanent_uuid: "0234bcd3c17148fe8b56675f98cd2c92" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 41481 } }
I20260807 08:47:37.290249 6286 raft_consensus.cc:348] T ce694c5f05cf43dfbdc689171f28ec71 P 9dadd97d8b3a4b01a5e53105e077f90a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "734acbba8fe5459db24d0b3422150dc3" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 39613 } } peers { permanent_uuid: "9dadd97d8b3a4b01a5e53105e077f90a" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 35331 } } peers { permanent_uuid: "0234bcd3c17148fe8b56675f98cd2c92" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 41481 } }
I20260807 08:47:37.290443 6286 raft_consensus.cc:374] T ce694c5f05cf43dfbdc689171f28ec71 P 9dadd97d8b3a4b01a5e53105e077f90a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:47:37.290488 6286 raft_consensus.cc:729] T ce694c5f05cf43dfbdc689171f28ec71 P 9dadd97d8b3a4b01a5e53105e077f90a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9dadd97d8b3a4b01a5e53105e077f90a, State: Initialized, Role: FOLLOWER
I20260807 08:47:37.290506 6284 ts_tablet_manager.cc:1428] T ce694c5f05cf43dfbdc689171f28ec71 P 0234bcd3c17148fe8b56675f98cd2c92: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 08:47:37.290622 6286 consensus_queue.cc:260] T ce694c5f05cf43dfbdc689171f28ec71 P 9dadd97d8b3a4b01a5e53105e077f90a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "734acbba8fe5459db24d0b3422150dc3" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 39613 } } peers { permanent_uuid: "9dadd97d8b3a4b01a5e53105e077f90a" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 35331 } } peers { permanent_uuid: "0234bcd3c17148fe8b56675f98cd2c92" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 41481 } }
I20260807 08:47:37.290704 5880 heartbeater.cc:503] Master 127.4.159.190:36157 was elected leader, sending a full tablet report...
I20260807 08:47:37.290911 6269 heartbeater.cc:503] Master 127.4.159.190:36157 was elected leader, sending a full tablet report...
I20260807 08:47:37.290917 6286 ts_tablet_manager.cc:1428] T ce694c5f05cf43dfbdc689171f28ec71 P 9dadd97d8b3a4b01a5e53105e077f90a: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260807 08:47:37.292114 6287 raft_consensus.cc:348] T ce694c5f05cf43dfbdc689171f28ec71 P 734acbba8fe5459db24d0b3422150dc3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "734acbba8fe5459db24d0b3422150dc3" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 39613 } } peers { permanent_uuid: "9dadd97d8b3a4b01a5e53105e077f90a" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 35331 } } peers { permanent_uuid: "0234bcd3c17148fe8b56675f98cd2c92" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 41481 } }
I20260807 08:47:37.292245 6287 raft_consensus.cc:374] T ce694c5f05cf43dfbdc689171f28ec71 P 734acbba8fe5459db24d0b3422150dc3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:47:37.292279 6287 raft_consensus.cc:729] T ce694c5f05cf43dfbdc689171f28ec71 P 734acbba8fe5459db24d0b3422150dc3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 734acbba8fe5459db24d0b3422150dc3, State: Initialized, Role: FOLLOWER
I20260807 08:47:37.292382 6287 consensus_queue.cc:260] T ce694c5f05cf43dfbdc689171f28ec71 P 734acbba8fe5459db24d0b3422150dc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "734acbba8fe5459db24d0b3422150dc3" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 39613 } } peers { permanent_uuid: "9dadd97d8b3a4b01a5e53105e077f90a" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 35331 } } peers { permanent_uuid: "0234bcd3c17148fe8b56675f98cd2c92" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 41481 } }
I20260807 08:47:37.292691 6072 heartbeater.cc:503] Master 127.4.159.190:36157 was elected leader, sending a full tablet report...
I20260807 08:47:37.292974 6287 ts_tablet_manager.cc:1428] T ce694c5f05cf43dfbdc689171f28ec71 P 734acbba8fe5459db24d0b3422150dc3: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 08:47:37.329681 6293 raft_consensus.cc:482] T ce694c5f05cf43dfbdc689171f28ec71 P 734acbba8fe5459db24d0b3422150dc3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:47:37.329787 6293 raft_consensus.cc:504] T ce694c5f05cf43dfbdc689171f28ec71 P 734acbba8fe5459db24d0b3422150dc3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "734acbba8fe5459db24d0b3422150dc3" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 39613 } } peers { permanent_uuid: "9dadd97d8b3a4b01a5e53105e077f90a" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 35331 } } peers { permanent_uuid: "0234bcd3c17148fe8b56675f98cd2c92" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 41481 } }
I20260807 08:47:37.330358 6293 leader_election.cc:288] T ce694c5f05cf43dfbdc689171f28ec71 P 734acbba8fe5459db24d0b3422150dc3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9dadd97d8b3a4b01a5e53105e077f90a (127.4.159.131:35331), 0234bcd3c17148fe8b56675f98cd2c92 (127.4.159.129:41481)
I20260807 08:47:37.333940 6281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:37.330567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.130:32981 (local address 127.4.159.131:35331)
0807 08:47:37.330616 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:37.330618 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:37.330627 (+ 9us) server_negotiation.cc:407] Connection header received
0807 08:47:37.330655 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:37.330656 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:37.330665 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:37.330717 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:47:37.330895 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:37.331628 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:37.333009 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:37.333567 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:37.333636 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:37.333645 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:37.333837 (+ 192us) server_negotiation.cc:299] Negotiation successful
0807 08:47:37.333871 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260807 08:47:37.333940 6283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:37.330689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.130:33787 (local address 127.4.159.129:41481)
0807 08:47:37.330739 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:37.330741 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:37.330774 (+ 33us) server_negotiation.cc:407] Connection header received
0807 08:47:37.330836 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:37.330837 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:37.330847 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:37.330894 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:47:37.331021 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:37.331699 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:37.333012 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:37.333568 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:37.333637 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:37.333645 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:37.333833 (+ 188us) server_negotiation.cc:299] Negotiation successful
0807 08:47:37.333871 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260807 08:47:37.333961 6075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:37.330493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.131:35331 (local address 127.4.159.130:32981)
0807 08:47:37.330542 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0807 08:47:37.330554 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 08:47:37.330603 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:47:37.330749 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:47:37.330750 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:47:37.330752 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:47:37.330832 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:37.330834 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:37.331708 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:37.331710 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:37.332924 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:37.332927 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:37.333650 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:37.333651 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:37.333766 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:37.333769 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 08:47:37.333818 (+ 49us) client_negotiation.cc:240] Negotiation successful
0807 08:47:37.333867 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26,"mutex_wait_us":26}
I20260807 08:47:37.333967 6296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:37.330588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.129:41481 (local address 127.4.159.130:33787)
0807 08:47:37.330726 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0807 08:47:37.330735 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 08:47:37.330781 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:47:37.330913 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:47:37.330914 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:47:37.330915 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:47:37.330967 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:37.330968 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:37.331757 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:37.331758 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:37.332924 (+ 1166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:37.332927 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:37.333650 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:37.333651 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:37.333764 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:37.333769 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 08:47:37.333818 (+ 49us) client_negotiation.cc:240] Negotiation successful
0807 08:47:37.333867 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"mutex_wait_us":11,"thread_start_us":87,"threads_started":1}
I20260807 08:47:37.334421 6196 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce694c5f05cf43dfbdc689171f28ec71" candidate_uuid: "734acbba8fe5459db24d0b3422150dc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9dadd97d8b3a4b01a5e53105e077f90a" is_pre_election: true
I20260807 08:47:37.334575 6196 raft_consensus.cc:2454] T ce694c5f05cf43dfbdc689171f28ec71 P 9dadd97d8b3a4b01a5e53105e077f90a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 734acbba8fe5459db24d0b3422150dc3 in term 0.
I20260807 08:47:37.334846 5907 leader_election.cc:302] T ce694c5f05cf43dfbdc689171f28ec71 P 734acbba8fe5459db24d0b3422150dc3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 734acbba8fe5459db24d0b3422150dc3, 9dadd97d8b3a4b01a5e53105e077f90a; no voters:
I20260807 08:47:37.334973 6293 raft_consensus.cc:2781] T ce694c5f05cf43dfbdc689171f28ec71 P 734acbba8fe5459db24d0b3422150dc3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:47:37.335024 6293 raft_consensus.cc:482] T ce694c5f05cf43dfbdc689171f28ec71 P 734acbba8fe5459db24d0b3422150dc3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:47:37.335043 6293 raft_consensus.cc:3037] T ce694c5f05cf43dfbdc689171f28ec71 P 734acbba8fe5459db24d0b3422150dc3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:47:37.335693 6293 raft_consensus.cc:504] T ce694c5f05cf43dfbdc689171f28ec71 P 734acbba8fe5459db24d0b3422150dc3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "734acbba8fe5459db24d0b3422150dc3" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 39613 } } peers { permanent_uuid: "9dadd97d8b3a4b01a5e53105e077f90a" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 35331 } } peers { permanent_uuid: "0234bcd3c17148fe8b56675f98cd2c92" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 41481 } }
I20260807 08:47:37.335791 6293 leader_election.cc:288] T ce694c5f05cf43dfbdc689171f28ec71 P 734acbba8fe5459db24d0b3422150dc3 [CANDIDATE]: Term 1 election: Requested vote from peers 9dadd97d8b3a4b01a5e53105e077f90a (127.4.159.131:35331), 0234bcd3c17148fe8b56675f98cd2c92 (127.4.159.129:41481)
I20260807 08:47:37.335927 6196 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce694c5f05cf43dfbdc689171f28ec71" candidate_uuid: "734acbba8fe5459db24d0b3422150dc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9dadd97d8b3a4b01a5e53105e077f90a"
I20260807 08:47:37.336011 6196 raft_consensus.cc:3037] T ce694c5f05cf43dfbdc689171f28ec71 P 9dadd97d8b3a4b01a5e53105e077f90a [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:47:37.336002 5817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce694c5f05cf43dfbdc689171f28ec71" candidate_uuid: "734acbba8fe5459db24d0b3422150dc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0234bcd3c17148fe8b56675f98cd2c92"
I20260807 08:47:37.336109 5817 raft_consensus.cc:3037] T ce694c5f05cf43dfbdc689171f28ec71 P 0234bcd3c17148fe8b56675f98cd2c92 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:47:37.336676 5818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce694c5f05cf43dfbdc689171f28ec71" candidate_uuid: "734acbba8fe5459db24d0b3422150dc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0234bcd3c17148fe8b56675f98cd2c92" is_pre_election: true
I20260807 08:47:37.336756 5817 raft_consensus.cc:2454] T ce694c5f05cf43dfbdc689171f28ec71 P 0234bcd3c17148fe8b56675f98cd2c92 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 734acbba8fe5459db24d0b3422150dc3 in term 1.
I20260807 08:47:37.336774 6196 raft_consensus.cc:2454] T ce694c5f05cf43dfbdc689171f28ec71 P 9dadd97d8b3a4b01a5e53105e077f90a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 734acbba8fe5459db24d0b3422150dc3 in term 1.
I20260807 08:47:37.336938 5907 leader_election.cc:302] T ce694c5f05cf43dfbdc689171f28ec71 P 734acbba8fe5459db24d0b3422150dc3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0234bcd3c17148fe8b56675f98cd2c92, 734acbba8fe5459db24d0b3422150dc3; no voters:
I20260807 08:47:37.337034 6293 raft_consensus.cc:2781] T ce694c5f05cf43dfbdc689171f28ec71 P 734acbba8fe5459db24d0b3422150dc3 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:47:37.337183 6293 raft_consensus.cc:686] T ce694c5f05cf43dfbdc689171f28ec71 P 734acbba8fe5459db24d0b3422150dc3 [term 1 LEADER]: Becoming Leader. State: Replica: 734acbba8fe5459db24d0b3422150dc3, State: Running, Role: LEADER
I20260807 08:47:37.337276 6293 consensus_queue.cc:237] T ce694c5f05cf43dfbdc689171f28ec71 P 734acbba8fe5459db24d0b3422150dc3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "734acbba8fe5459db24d0b3422150dc3" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 39613 } } peers { permanent_uuid: "9dadd97d8b3a4b01a5e53105e077f90a" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 35331 } } peers { permanent_uuid: "0234bcd3c17148fe8b56675f98cd2c92" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 41481 } }
I20260807 08:47:37.338249 5605 catalog_manager.cc:5495] T ce694c5f05cf43dfbdc689171f28ec71 P 734acbba8fe5459db24d0b3422150dc3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 734acbba8fe5459db24d0b3422150dc3 (127.4.159.130). New cstate: current_term: 1 leader_uuid: "734acbba8fe5459db24d0b3422150dc3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "734acbba8fe5459db24d0b3422150dc3" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 39613 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9dadd97d8b3a4b01a5e53105e077f90a" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 35331 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0234bcd3c17148fe8b56675f98cd2c92" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 41481 } health_report { overall_health: UNKNOWN } } }
W20260807 08:47:37.352247 6073 tablet.cc:2369] T ce694c5f05cf43dfbdc689171f28ec71 P 734acbba8fe5459db24d0b3422150dc3: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 07 08:47:37 dist-test-slave-6zbq krb5kdc[5544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092456, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.159.129@KRBTEST.COM
I20260807 08:47:37.360028 6283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:37.353852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59768 (local address 127.4.159.129:41481)
0807 08:47:37.353922 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:37.353924 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:37.353973 (+ 49us) server_negotiation.cc:407] Connection header received
0807 08:47:37.354016 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:37.354017 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:37.354025 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:37.354067 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:47:37.354565 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:37.355000 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:37.355613 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:37.355779 (+ 166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:37.355825 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:37.357865 (+ 2040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:47:37.357875 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:47:37.357875 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:47:37.357881 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:47:37.359201 (+ 1320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:37.359485 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:37.359488 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:37.359488 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:37.359538 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:37.359663 (+ 125us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:37.359664 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:37.359664 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:37.359772 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:47:37.359880 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:37.359906 (+ 26us) server_negotiation.cc:299] Negotiation successful
0807 08:47:37.359962 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
Aug 07 08:47:37 dist-test-slave-6zbq krb5kdc[5544](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092457, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 08:47:37 dist-test-slave-6zbq krb5kdc[5544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092457, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.4.159.129@KRBTEST.COM
I20260807 08:47:37.381834 6283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:37.375911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59776 (local address 127.4.159.129:41481)
0807 08:47:37.375969 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:37.375971 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:37.376034 (+ 63us) server_negotiation.cc:407] Connection header received
0807 08:47:37.376091 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:37.376092 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:37.376101 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:37.376145 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:47:37.376729 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:37.377184 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:37.377955 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:37.378132 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:37.378179 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:37.380008 (+ 1829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:47:37.380009 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:47:37.380010 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:47:37.380016 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:47:37.381007 (+ 991us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:37.381285 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:37.381285 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:37.381285 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:37.381329 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:37.381460 (+ 131us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:37.381461 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:37.381461 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:37.381539 (+ 78us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:47:37.381607 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:37.381712 (+ 105us) server_negotiation.cc:299] Negotiation successful
0807 08:47:37.381767 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
Aug 07 08:47:37 dist-test-slave-6zbq krb5kdc[5544](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092457, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 08:47:37 dist-test-slave-6zbq krb5kdc[5544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092457, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.159.129@KRBTEST.COM
I20260807 08:47:37.402568 6283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:37.396038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59784 (local address 127.4.159.129:41481)
0807 08:47:37.396087 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:37.396088 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:37.396761 (+ 673us) server_negotiation.cc:407] Connection header received
0807 08:47:37.396784 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:37.396785 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:37.396796 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:37.396894 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:47:37.397473 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:37.397888 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:37.398572 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:37.398744 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:37.398786 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:37.400559 (+ 1773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:47:37.400560 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:47:37.400560 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:47:37.400566 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:47:37.401700 (+ 1134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:37.401973 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:37.401974 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:37.401974 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:37.402028 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:37.402177 (+ 149us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:37.402177 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:37.402178 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:37.402253 (+ 75us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:47:37.402311 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:37.402450 (+ 139us) server_negotiation.cc:299] Negotiation successful
0807 08:47:37.402503 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
W20260807 08:47:37.403204 5753 tablet_service.cc:2908] Rejecting scan request for tablet ce694c5f05cf43dfbdc689171f28ec71: Uninitialized: clean time has not yet been initialized
I20260807 08:47:37.403966 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 5698
I20260807 08:47:37.413738 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 5888
I20260807 08:47:37.424623 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 6078
I20260807 08:47:37.431144 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 5563
2026-08-07T08:47:37Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (1960 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786092453486433-4734-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:47:37 dist-test-slave-6zbq krb5kdc[6315](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:47:37 dist-test-slave-6zbq krb5kdc[6315](info): set up 2 sockets
Aug 07 08:47:37 dist-test-slave-6zbq krb5kdc[6315](info): commencing operation
krb5kdc: starting...
W20260807 08:47:38.565474 4734 mini_kdc.cc:121] Time spent starting KDC: real 1.113s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786092453486433-4734-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786092453486433-4734-0/krb5kdc/test-user.keytab.
Aug 07 08:47:38 dist-test-slave-6zbq krb5kdc[6315](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092458, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:47:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:47:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.159.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.159.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.159.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab.
Entry for principal kudu/127.4.159.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab.
I20260807 08:47:38.674193 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786092453486433-4734-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786092453486433-4734-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786092453486433-4734-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.159.190:43663
--webserver_interface=127.4.159.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab
--principal=kudu/127.4.159.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:33127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.159.190:43663
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:47:38.752262 6331 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:47:38.752441 6331 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:47:38.752461 6331 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:47:38.753510 6331 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:47:38.753540 6331 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 08:47:38.753551 6331 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:47:38.753562 6331 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:47:38.753573 6331 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:47:38.753583 6331 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:47:38.754999 6331 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:33127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786092453486433-4734-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786092453486433-4734-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.159.190:43663
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab
--principal=kudu/127.4.159.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.159.190:43663
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.159.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6331
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786092453486433-4734-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:47:38.755189 6331 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:47:38.755343 6331 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:47:38.759740 6337 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:47:38.759742 6345 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:47:38.759766 6342 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:47:38.759974 6331 server_base.cc:1034] running on GCE node
I20260807 08:47:38.760180 6331 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:47:38.760445 6331 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:47:38.761580 6331 hybrid_clock.cc:648] HybridClock initialized: now 1786092458761566 us; error 28 us; skew 500 ppm
Aug 07 08:47:38 dist-test-slave-6zbq krb5kdc[6315](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092458, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:47:38.768115 6331 init.cc:377] Logged in from keytab as kudu/127.4.159.190@KRBTEST.COM (short username kudu)
I20260807 08:47:38.768510 6331 webserver.cc:460] Webserver started at http://127.4.159.190:46675/ using document root <none> and password file <none>
I20260807 08:47:38.768694 6331 fs_manager.cc:359] Metadata directory not provided
I20260807 08:47:38.768739 6331 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:47:38.768868 6331 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:47:38.769822 6331 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786092453486433-4734-0/minicluster-data/master-0/data/instance:
uuid: "a9b26c4a4f35408cac8649134220bab5"
format_stamp: "Formatted at 2026-08-07 08:47:38 on dist-test-slave-6zbq"
server_key: "09eeba2e83bf25a243e0c1fa01cf4620"
server_key_iv: "783d4628071b064389097394be64fd1a"
server_key_version: "encryptionkey@0"
I20260807 08:47:38.770136 6331 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786092453486433-4734-0/minicluster-data/master-0/wal/instance:
uuid: "a9b26c4a4f35408cac8649134220bab5"
format_stamp: "Formatted at 2026-08-07 08:47:38 on dist-test-slave-6zbq"
server_key: "09eeba2e83bf25a243e0c1fa01cf4620"
server_key_iv: "783d4628071b064389097394be64fd1a"
server_key_version: "encryptionkey@0"
I20260807 08:47:38.771453 6331 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 08:47:38.772104 6356 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:38.772250 6331 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 08:47:38.772331 6331 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786092453486433-4734-0/minicluster-data/master-0/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786092453486433-4734-0/minicluster-data/master-0/wal
uuid: "a9b26c4a4f35408cac8649134220bab5"
format_stamp: "Formatted at 2026-08-07 08:47:38 on dist-test-slave-6zbq"
server_key: "09eeba2e83bf25a243e0c1fa01cf4620"
server_key_iv: "783d4628071b064389097394be64fd1a"
server_key_version: "encryptionkey@0"
I20260807 08:47:38.772397 6331 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786092453486433-4734-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786092453486433-4734-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786092453486433-4734-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:47:38.797163 6331 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:47:38.798815 6331 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:47:38.798938 6331 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:47:38.803359 6331 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.190:43663
I20260807 08:47:38.803414 6454 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.190:43663 every 8 connection(s)
I20260807 08:47:38.803676 6331 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
I20260807 08:47:38.804250 6455 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:47:38.806437 6455 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9b26c4a4f35408cac8649134220bab5: Bootstrap starting.
I20260807 08:47:38.807000 6455 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a9b26c4a4f35408cac8649134220bab5: Neither blocks nor log segments found. Creating new log.
I20260807 08:47:38.807266 6455 log.cc:824] T 00000000000000000000000000000000 P a9b26c4a4f35408cac8649134220bab5: Log is configured to *not* fsync() on all Append() calls
I20260807 08:47:38.807894 6455 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9b26c4a4f35408cac8649134220bab5: No bootstrap required, opened a new log
I20260807 08:47:38.808202 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 6331
I20260807 08:47:38.808297 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786092453486433-4734-0/minicluster-data/master-0/wal/instance
I20260807 08:47:38.808457 4734 external_mini_cluster.cc:1428] Setting key 23c49004a9950f8869caebd02be56c0a
I20260807 08:47:38.810390 6455 raft_consensus.cc:348] T 00000000000000000000000000000000 P a9b26c4a4f35408cac8649134220bab5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9b26c4a4f35408cac8649134220bab5" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 43663 } }
I20260807 08:47:38.810523 6455 raft_consensus.cc:374] T 00000000000000000000000000000000 P a9b26c4a4f35408cac8649134220bab5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:47:38.810544 6455 raft_consensus.cc:729] T 00000000000000000000000000000000 P a9b26c4a4f35408cac8649134220bab5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9b26c4a4f35408cac8649134220bab5, State: Initialized, Role: FOLLOWER
I20260807 08:47:38.810648 6455 consensus_queue.cc:260] T 00000000000000000000000000000000 P a9b26c4a4f35408cac8649134220bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9b26c4a4f35408cac8649134220bab5" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 43663 } }
I20260807 08:47:38.810710 6455 raft_consensus.cc:388] T 00000000000000000000000000000000 P a9b26c4a4f35408cac8649134220bab5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:47:38.810731 6455 raft_consensus.cc:482] T 00000000000000000000000000000000 P a9b26c4a4f35408cac8649134220bab5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:47:38.810775 6455 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a9b26c4a4f35408cac8649134220bab5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:47:38.811260 6455 raft_consensus.cc:504] T 00000000000000000000000000000000 P a9b26c4a4f35408cac8649134220bab5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9b26c4a4f35408cac8649134220bab5" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 43663 } }
I20260807 08:47:38.811369 6455 leader_election.cc:302] T 00000000000000000000000000000000 P a9b26c4a4f35408cac8649134220bab5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a9b26c4a4f35408cac8649134220bab5; no voters:
I20260807 08:47:38.811525 6455 leader_election.cc:288] T 00000000000000000000000000000000 P a9b26c4a4f35408cac8649134220bab5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:47:38.811581 6461 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a9b26c4a4f35408cac8649134220bab5 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:47:38.811700 6461 raft_consensus.cc:686] T 00000000000000000000000000000000 P a9b26c4a4f35408cac8649134220bab5 [term 1 LEADER]: Becoming Leader. State: Replica: a9b26c4a4f35408cac8649134220bab5, State: Running, Role: LEADER
I20260807 08:47:38.811771 6455 sys_catalog.cc:564] T 00000000000000000000000000000000 P a9b26c4a4f35408cac8649134220bab5 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:47:38.811782 6461 consensus_queue.cc:237] T 00000000000000000000000000000000 P a9b26c4a4f35408cac8649134220bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9b26c4a4f35408cac8649134220bab5" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 43663 } }
I20260807 08:47:38.812974 6463 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9b26c4a4f35408cac8649134220bab5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a9b26c4a4f35408cac8649134220bab5. Latest consensus state: current_term: 1 leader_uuid: "a9b26c4a4f35408cac8649134220bab5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9b26c4a4f35408cac8649134220bab5" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 43663 } } }
I20260807 08:47:38.813053 6462 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9b26c4a4f35408cac8649134220bab5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a9b26c4a4f35408cac8649134220bab5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9b26c4a4f35408cac8649134220bab5" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 43663 } } }
I20260807 08:47:38.813061 6463 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9b26c4a4f35408cac8649134220bab5 [sys.catalog]: This master's current role is: LEADER
I20260807 08:47:38.813153 6462 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9b26c4a4f35408cac8649134220bab5 [sys.catalog]: This master's current role is: LEADER
I20260807 08:47:38.813378 6480 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:47:38.814002 6480 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 07 08:47:38 dist-test-slave-6zbq krb5kdc[6315](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092458, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.159.190@KRBTEST.COM
I20260807 08:47:38.815611 6480 catalog_manager.cc:1347] Generated new cluster ID: 0d9b11e5b6e0449da751c39e3c3c5dba
I20260807 08:47:38.815665 6480 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:47:38.818792 6460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:38.808906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35640 (local address 127.4.159.190:43663)
0807 08:47:38.809077 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:38.809086 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:38.809110 (+ 24us) server_negotiation.cc:407] Connection header received
0807 08:47:38.810039 (+ 929us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:38.810047 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:38.810096 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:38.810217 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:47:38.810918 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:38.811979 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:38.812972 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:38.813544 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:38.813658 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:38.815952 (+ 2294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:47:38.815957 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:47:38.815957 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:47:38.815969 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:47:38.817422 (+ 1453us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:38.817735 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:38.817738 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:38.817739 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:38.817834 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:38.818010 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:38.818010 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:38.818011 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:38.818229 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:47:38.818399 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:38.818490 (+ 91us) server_negotiation.cc:299] Negotiation successful
0807 08:47:38.818626 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":74,"threads_started":1}
I20260807 08:47:38.870327 6480 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:47:38.870873 6480 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:47:38.875270 6480 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a9b26c4a4f35408cac8649134220bab5: Generated new TSK 0
I20260807 08:47:38.875430 6480 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.159.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.159.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.159.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.129.keytab.
Entry for principal kudu/127.4.159.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.129.keytab.
I20260807 08:47:38.905676 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786092453486433-4734-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786092453486433-4734-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786092453486433-4734-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786092453486433-4734-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.159.129:0
--local_ip_for_outbound_sockets=127.4.159.129
--webserver_interface=127.4.159.129
--webserver_port=0
--tserver_master_addrs=127.4.159.190:43663
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.129.keytab
--principal=kudu/127.4.159.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:33127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:47:38.983256 6487 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:47:38.983435 6487 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:47:38.983464 6487 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:47:38.984572 6487 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:47:38.984611 6487 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:47:38.984642 6487 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 08:47:38.984666 6487 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.159.129
I20260807 08:47:38.986027 6487 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:33127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786092453486433-4734-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786092453486433-4734-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.129.keytab
--principal=kudu/127.4.159.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.159.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786092453486433-4734-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.159.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.4.159.190:43663
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6487
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.159.129
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786092453486433-4734-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:47:38.986248 6487 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:47:38.986465 6487 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:47:38.990901 6497 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:47:38.990875 6495 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:47:38.991019 6487 server_base.cc:1034] running on GCE node
W20260807 08:47:38.990875 6499 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:47:38.991285 6487 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:47:38.991508 6487 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:47:38.992642 6487 hybrid_clock.cc:648] HybridClock initialized: now 1786092458992628 us; error 27 us; skew 500 ppm
Aug 07 08:47:38 dist-test-slave-6zbq krb5kdc[6315](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092458, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:47:38.994951 6487 init.cc:377] Logged in from keytab as kudu/127.4.159.129@KRBTEST.COM (short username kudu)
I20260807 08:47:38.995344 6487 webserver.cc:460] Webserver started at http://127.4.159.129:45113/ using document root <none> and password file <none>
I20260807 08:47:38.995524 6487 fs_manager.cc:359] Metadata directory not provided
I20260807 08:47:38.995565 6487 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:47:38.995682 6487 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:47:38.996598 6487 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786092453486433-4734-0/minicluster-data/ts-0/data/instance:
uuid: "48b5111c255f4226b5ce348cc76a2d1b"
format_stamp: "Formatted at 2026-08-07 08:47:38 on dist-test-slave-6zbq"
server_key: "6b7c39dd1f1b41b46a3162c5922fa7d0"
server_key_iv: "da8f4c8e0f58c392b3a8445b535ae89e"
server_key_version: "encryptionkey@0"
I20260807 08:47:38.996903 6487 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786092453486433-4734-0/minicluster-data/ts-0/wal/instance:
uuid: "48b5111c255f4226b5ce348cc76a2d1b"
format_stamp: "Formatted at 2026-08-07 08:47:38 on dist-test-slave-6zbq"
server_key: "6b7c39dd1f1b41b46a3162c5922fa7d0"
server_key_iv: "da8f4c8e0f58c392b3a8445b535ae89e"
server_key_version: "encryptionkey@0"
I20260807 08:47:38.998073 6487 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 08:47:38.998736 6507 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:38.998898 6487 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:38.998979 6487 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786092453486433-4734-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786092453486433-4734-0/minicluster-data/ts-0/wal
uuid: "48b5111c255f4226b5ce348cc76a2d1b"
format_stamp: "Formatted at 2026-08-07 08:47:38 on dist-test-slave-6zbq"
server_key: "6b7c39dd1f1b41b46a3162c5922fa7d0"
server_key_iv: "da8f4c8e0f58c392b3a8445b535ae89e"
server_key_version: "encryptionkey@0"
I20260807 08:47:38.999034 6487 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786092453486433-4734-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786092453486433-4734-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786092453486433-4734-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:47:39.010995 6487 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:47:39.012733 6487 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:47:39.012873 6487 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:47:39.013325 6487 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:47:39.013362 6487 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:39.013383 6487 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:47:39.013406 6487 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:39.019596 6487 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.129:44539
I20260807 08:47:39.019688 6686 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.129:44539 every 8 connection(s)
Aug 07 08:47:39 dist-test-slave-6zbq krb5kdc[6315](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092458, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.129@KRBTEST.COM for kudu/127.4.159.190@KRBTEST.COM
I20260807 08:47:39.019951 6487 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786092453486433-4734-0/minicluster-data/ts-0/data/info.pb
I20260807 08:47:39.022436 6460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:39.013835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.129:59911 (local address 127.4.159.190:43663)
0807 08:47:39.013901 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:39.013903 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:39.014038 (+ 135us) server_negotiation.cc:407] Connection header received
0807 08:47:39.014545 (+ 507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:39.014546 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:39.014558 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:39.014632 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:47:39.015365 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:39.016113 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:39.017517 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:39.017819 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:39.017895 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:39.020202 (+ 2307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:47:39.020204 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:47:39.020204 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:47:39.020213 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:47:39.021395 (+ 1182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:39.021713 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:39.021713 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:39.021714 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:39.021765 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:39.021932 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:39.021932 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:39.021932 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:39.022022 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:47:39.022067 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:39.022248 (+ 181us) server_negotiation.cc:299] Negotiation successful
0807 08:47:39.022328 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260807 08:47:39.022523 6521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:39.013780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:43663 (local address 127.4.159.129:59911)
0807 08:47:39.013929 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0807 08:47:39.013966 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 08:47:39.014458 (+ 492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:47:39.014698 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:47:39.014701 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:47:39.014996 (+ 295us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:47:39.015251 (+ 255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:39.015257 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:39.016204 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:39.016205 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:39.017427 (+ 1222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:39.017430 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:39.017902 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:39.017902 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:39.018018 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:39.018459 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:47:39.018478 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:47:39.020110 (+ 1632us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:47:39.021458 (+ 1348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:47:39.021463 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:47:39.021465 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:47:39.021659 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:47:39.021808 (+ 149us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:47:39.021808 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:47:39.021808 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:47:39.021891 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:47:39.022062 (+ 171us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:47:39.022065 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:47:39.022159 (+ 94us) client_negotiation.cc:769] Sending connection context
0807 08:47:39.022229 (+ 70us) client_negotiation.cc:240] Negotiation successful
0807 08:47:39.022366 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":77,"threads_started":1}
I20260807 08:47:39.022984 6687 heartbeater.cc:347] Connected to a master server at 127.4.159.190:43663
I20260807 08:47:39.023069 6687 heartbeater.cc:464] Registering TS with master...
I20260807 08:47:39.023240 6687 heartbeater.cc:511] Master 127.4.159.190:43663 requested a full tablet report, sending...
I20260807 08:47:39.023725 6387 ts_manager.cc:194] Registered new tserver with Master: 48b5111c255f4226b5ce348cc76a2d1b (127.4.159.129:44539)
I20260807 08:47:39.024403 6387 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.159.129@KRBTEST.COM'} at 127.4.159.129:59911
I20260807 08:47:39.028437 6695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:39.023926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:43663 (local address 127.4.159.129:37491)
0807 08:47:39.024102 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0807 08:47:39.024111 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 08:47:39.024162 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:47:39.024278 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:47:39.024278 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:47:39.024440 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:47:39.024484 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:39.024485 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:39.025084 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:39.025085 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:39.025560 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:39.025562 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:39.025830 (+ 268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:39.025831 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:39.025879 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:39.026215 (+ 336us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:47:39.026220 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:47:39.026519 (+ 299us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:47:39.027666 (+ 1147us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:47:39.027667 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:47:39.027667 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:47:39.027811 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:47:39.027994 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:47:39.027995 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:47:39.027995 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:47:39.028037 (+ 42us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:47:39.028225 (+ 188us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:47:39.028225 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:47:39.028256 (+ 31us) client_negotiation.cc:769] Sending connection context
0807 08:47:39.028313 (+ 57us) client_negotiation.cc:240] Negotiation successful
0807 08:47:39.028377 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":121,"threads_started":1}
I20260807 08:47:39.028445 6460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:39.023972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.129:37491 (local address 127.4.159.190:43663)
0807 08:47:39.024005 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:39.024007 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:39.024161 (+ 154us) server_negotiation.cc:407] Connection header received
0807 08:47:39.024213 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:39.024214 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:39.024223 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:39.024274 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:47:39.024544 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:39.024998 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:39.025619 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:39.025784 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:39.025829 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:39.026583 (+ 754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:47:39.026584 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:47:39.026584 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:47:39.026588 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:47:39.027608 (+ 1020us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:39.027867 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:39.027867 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:39.027868 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:39.027946 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:39.028071 (+ 125us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:39.028071 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:39.028072 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:39.028183 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:47:39.028228 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:39.028339 (+ 111us) server_negotiation.cc:299] Negotiation successful
0807 08:47:39.028390 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260807 08:47:39.029814 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 6487
I20260807 08:47:39.029899 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786092453486433-4734-0/minicluster-data/ts-0/wal/instance
I20260807 08:47:39.030050 4734 external_mini_cluster.cc:1428] Setting key 415613f735316b9e401b48efb8058dfa
WARNING: no policy specified for kudu/127.4.159.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.159.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.159.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.130.keytab.
Entry for principal kudu/127.4.159.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.130.keytab.
I20260807 08:47:39.050695 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786092453486433-4734-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786092453486433-4734-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786092453486433-4734-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786092453486433-4734-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.159.130:0
--local_ip_for_outbound_sockets=127.4.159.130
--webserver_interface=127.4.159.130
--webserver_port=0
--tserver_master_addrs=127.4.159.190:43663
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.130.keytab
--principal=kudu/127.4.159.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:33127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:47:39.127772 6700 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:47:39.127941 6700 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:47:39.127962 6700 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:47:39.129055 6700 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:47:39.129092 6700 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:47:39.129113 6700 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 08:47:39.129144 6700 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.159.130
I20260807 08:47:39.130548 6700 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:33127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786092453486433-4734-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786092453486433-4734-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.130.keytab
--principal=kudu/127.4.159.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.159.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786092453486433-4734-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.159.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.4.159.190:43663
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6700
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.159.130
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786092453486433-4734-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:47:39.130746 6700 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:47:39.130937 6700 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:47:39.135370 6712 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:47:39.135387 6707 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:47:39.135383 6710 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:47:39.135583 6700 server_base.cc:1034] running on GCE node
I20260807 08:47:39.135738 6700 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:47:39.135934 6700 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:47:39.137058 6700 hybrid_clock.cc:648] HybridClock initialized: now 1786092459137051 us; error 27 us; skew 500 ppm
Aug 07 08:47:39 dist-test-slave-6zbq krb5kdc[6315](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092459, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:47:39.139274 6700 init.cc:377] Logged in from keytab as kudu/127.4.159.130@KRBTEST.COM (short username kudu)
I20260807 08:47:39.139640 6700 webserver.cc:460] Webserver started at http://127.4.159.130:34701/ using document root <none> and password file <none>
I20260807 08:47:39.139847 6700 fs_manager.cc:359] Metadata directory not provided
I20260807 08:47:39.139899 6700 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:47:39.140005 6700 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:47:39.140930 6700 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786092453486433-4734-0/minicluster-data/ts-1/data/instance:
uuid: "661ee1d16b2648d98bce6027b5132089"
format_stamp: "Formatted at 2026-08-07 08:47:39 on dist-test-slave-6zbq"
server_key: "534a2aee3b110b91bd02c462fdea750b"
server_key_iv: "a4ce04f4e1c9fdfda869880b8cfa9ee0"
server_key_version: "encryptionkey@0"
I20260807 08:47:39.141239 6700 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786092453486433-4734-0/minicluster-data/ts-1/wal/instance:
uuid: "661ee1d16b2648d98bce6027b5132089"
format_stamp: "Formatted at 2026-08-07 08:47:39 on dist-test-slave-6zbq"
server_key: "534a2aee3b110b91bd02c462fdea750b"
server_key_iv: "a4ce04f4e1c9fdfda869880b8cfa9ee0"
server_key_version: "encryptionkey@0"
I20260807 08:47:39.142427 6700 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 08:47:39.143100 6721 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:39.143244 6700 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 08:47:39.143319 6700 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786092453486433-4734-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786092453486433-4734-0/minicluster-data/ts-1/wal
uuid: "661ee1d16b2648d98bce6027b5132089"
format_stamp: "Formatted at 2026-08-07 08:47:39 on dist-test-slave-6zbq"
server_key: "534a2aee3b110b91bd02c462fdea750b"
server_key_iv: "a4ce04f4e1c9fdfda869880b8cfa9ee0"
server_key_version: "encryptionkey@0"
I20260807 08:47:39.143385 6700 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786092453486433-4734-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786092453486433-4734-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786092453486433-4734-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:47:39.165407 6700 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:47:39.167029 6700 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:47:39.167147 6700 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:47:39.167578 6700 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:47:39.167620 6700 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:39.167639 6700 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:47:39.167650 6700 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 08:47:39 dist-test-slave-6zbq krb5kdc[6315](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092459, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.130@KRBTEST.COM for kudu/127.4.159.190@KRBTEST.COM
I20260807 08:47:39.174139 6700 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.130:45177
I20260807 08:47:39.174193 6885 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.130:45177 every 8 connection(s)
I20260807 08:47:39.174579 6700 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786092453486433-4734-0/minicluster-data/ts-1/data/info.pb
I20260807 08:47:39.176657 6460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:39.167992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.130:43093 (local address 127.4.159.190:43663)
0807 08:47:39.168056 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:39.168058 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:39.168200 (+ 142us) server_negotiation.cc:407] Connection header received
0807 08:47:39.168632 (+ 432us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:39.168633 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:39.168647 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:39.168714 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:47:39.169445 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:39.170177 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:39.171179 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:39.171570 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:39.171645 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:39.174295 (+ 2650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:47:39.174297 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:47:39.174297 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:47:39.174305 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:47:39.175626 (+ 1321us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:39.175958 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:39.175959 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:39.175959 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:39.176003 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:39.176189 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:39.176190 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:39.176190 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:39.176298 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:47:39.176358 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:39.176539 (+ 181us) server_negotiation.cc:299] Negotiation successful
0807 08:47:39.176596 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260807 08:47:39.176810 6732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:39.167940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:43663 (local address 127.4.159.130:43093)
0807 08:47:39.168089 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0807 08:47:39.168126 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 08:47:39.168551 (+ 425us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:47:39.168776 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:47:39.168780 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:47:39.169080 (+ 300us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:47:39.169349 (+ 269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:39.169355 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:39.170296 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:39.170297 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:39.171109 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:39.171111 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:39.171662 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:39.171663 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:39.171813 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:39.172218 (+ 405us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:47:39.172228 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:47:39.174193 (+ 1965us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:47:39.175692 (+ 1499us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:47:39.175697 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:47:39.175699 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:47:39.175896 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:47:39.176043 (+ 147us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:47:39.176043 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:47:39.176043 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:47:39.176128 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:47:39.176353 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:47:39.176356 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:47:39.176452 (+ 96us) client_negotiation.cc:769] Sending connection context
0807 08:47:39.176524 (+ 72us) client_negotiation.cc:240] Negotiation successful
0807 08:47:39.176629 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":84,"threads_started":1}
I20260807 08:47:39.177238 6886 heartbeater.cc:347] Connected to a master server at 127.4.159.190:43663
I20260807 08:47:39.177317 6886 heartbeater.cc:464] Registering TS with master...
I20260807 08:47:39.177488 6886 heartbeater.cc:511] Master 127.4.159.190:43663 requested a full tablet report, sending...
I20260807 08:47:39.177847 6388 ts_manager.cc:194] Registered new tserver with Master: 661ee1d16b2648d98bce6027b5132089 (127.4.159.130:45177)
I20260807 08:47:39.178283 6388 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.159.130@KRBTEST.COM'} at 127.4.159.130:43093
I20260807 08:47:39.182574 6897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:39.178114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:43663 (local address 127.4.159.130:36365)
0807 08:47:39.178229 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0807 08:47:39.178239 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 08:47:39.178280 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:47:39.178548 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:47:39.178549 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:47:39.178730 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:47:39.178810 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:39.178812 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:39.179384 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:39.179385 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:39.179855 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:39.179857 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:39.180113 (+ 256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:39.180114 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:39.180162 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:39.180498 (+ 336us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:47:39.180503 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:47:39.180743 (+ 240us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:47:39.181845 (+ 1102us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:47:39.181846 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:47:39.181846 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:47:39.181984 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:47:39.182106 (+ 122us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:47:39.182107 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:47:39.182107 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:47:39.182145 (+ 38us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:47:39.182330 (+ 185us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:47:39.182330 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:47:39.182393 (+ 63us) client_negotiation.cc:769] Sending connection context
0807 08:47:39.182445 (+ 52us) client_negotiation.cc:240] Negotiation successful
0807 08:47:39.182508 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":54,"threads_started":1}
I20260807 08:47:39.182582 6460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:39.178404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.130:36365 (local address 127.4.159.190:43663)
0807 08:47:39.178442 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:39.178444 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:39.178451 (+ 7us) server_negotiation.cc:407] Connection header received
0807 08:47:39.178470 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:39.178471 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:39.178479 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:39.178531 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:47:39.178881 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:39.179304 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:39.179910 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:39.180068 (+ 158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:39.180107 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:39.180824 (+ 717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:47:39.180824 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:47:39.180825 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:47:39.180829 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:47:39.181784 (+ 955us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:39.182031 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:39.182032 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:39.182032 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:39.182072 (+ 40us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:39.182176 (+ 104us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:39.182176 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:39.182177 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:39.182288 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:47:39.182335 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:39.182472 (+ 137us) server_negotiation.cc:299] Negotiation successful
0807 08:47:39.182528 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 08:47:39.184705 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 6700
I20260807 08:47:39.184777 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786092453486433-4734-0/minicluster-data/ts-1/wal/instance
I20260807 08:47:39.184928 4734 external_mini_cluster.cc:1428] Setting key 796000c4113b21bb9728ee48d7c05f21
WARNING: no policy specified for kudu/127.4.159.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.159.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.159.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.131.keytab.
Entry for principal kudu/127.4.159.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.131.keytab.
I20260807 08:47:39.206321 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786092453486433-4734-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786092453486433-4734-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786092453486433-4734-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786092453486433-4734-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.159.131:0
--local_ip_for_outbound_sockets=127.4.159.131
--webserver_interface=127.4.159.131
--webserver_port=0
--tserver_master_addrs=127.4.159.190:43663
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.131.keytab
--principal=kudu/127.4.159.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:33127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:47:39.284317 6903 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:47:39.284503 6903 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:47:39.284534 6903 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:47:39.285658 6903 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:47:39.285698 6903 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:47:39.285725 6903 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 08:47:39.285748 6903 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.159.131
I20260807 08:47:39.287143 6903 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:33127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786092453486433-4734-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786092453486433-4734-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.131.keytab
--principal=kudu/127.4.159.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.159.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786092453486433-4734-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.159.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.4.159.190:43663
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6903
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.159.131
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786092453486433-4734-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:47:39.287360 6903 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:47:39.287561 6903 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:47:39.292188 6910 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:47:39.292182 6912 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:47:39.292243 6903 server_base.cc:1034] running on GCE node
W20260807 08:47:39.292186 6914 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:47:39.292572 6903 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:47:39.292804 6903 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:47:39.293936 6903 hybrid_clock.cc:648] HybridClock initialized: now 1786092459293925 us; error 28 us; skew 500 ppm
Aug 07 08:47:39 dist-test-slave-6zbq krb5kdc[6315](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092459, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:47:39.296201 6903 init.cc:377] Logged in from keytab as kudu/127.4.159.131@KRBTEST.COM (short username kudu)
I20260807 08:47:39.296607 6903 webserver.cc:460] Webserver started at http://127.4.159.131:40733/ using document root <none> and password file <none>
I20260807 08:47:39.296793 6903 fs_manager.cc:359] Metadata directory not provided
I20260807 08:47:39.296835 6903 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:47:39.296954 6903 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:47:39.297910 6903 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786092453486433-4734-0/minicluster-data/ts-2/data/instance:
uuid: "2348d0b1f3294f8a90178c62af62f40b"
format_stamp: "Formatted at 2026-08-07 08:47:39 on dist-test-slave-6zbq"
server_key: "b3244ce65c46c1dc6ed8a650378c3105"
server_key_iv: "bdbc51ae7e24f7fb52da56789d875637"
server_key_version: "encryptionkey@0"
I20260807 08:47:39.298219 6903 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786092453486433-4734-0/minicluster-data/ts-2/wal/instance:
uuid: "2348d0b1f3294f8a90178c62af62f40b"
format_stamp: "Formatted at 2026-08-07 08:47:39 on dist-test-slave-6zbq"
server_key: "b3244ce65c46c1dc6ed8a650378c3105"
server_key_iv: "bdbc51ae7e24f7fb52da56789d875637"
server_key_version: "encryptionkey@0"
I20260807 08:47:39.299387 6903 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 08:47:39.300079 6923 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:39.300225 6903 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 08:47:39.300308 6903 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786092453486433-4734-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786092453486433-4734-0/minicluster-data/ts-2/wal
uuid: "2348d0b1f3294f8a90178c62af62f40b"
format_stamp: "Formatted at 2026-08-07 08:47:39 on dist-test-slave-6zbq"
server_key: "b3244ce65c46c1dc6ed8a650378c3105"
server_key_iv: "bdbc51ae7e24f7fb52da56789d875637"
server_key_version: "encryptionkey@0"
I20260807 08:47:39.300376 6903 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786092453486433-4734-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786092453486433-4734-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786092453486433-4734-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:47:39.309473 6903 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:47:39.311462 6903 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:47:39.311583 6903 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:47:39.312076 6903 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:47:39.312119 6903 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:39.312139 6903 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:47:39.312151 6903 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 08:47:39 dist-test-slave-6zbq krb5kdc[6315](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092459, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.131@KRBTEST.COM for kudu/127.4.159.190@KRBTEST.COM
I20260807 08:47:39.318985 6903 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.131:35523
I20260807 08:47:39.319315 6903 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786092453486433-4734-0/minicluster-data/ts-2/data/info.pb
I20260807 08:47:39.319798 7088 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.131:35523 every 8 connection(s)
I20260807 08:47:39.320206 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 6903
I20260807 08:47:39.320294 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786092453486433-4734-0/minicluster-data/ts-2/wal/instance
I20260807 08:47:39.320456 4734 external_mini_cluster.cc:1428] Setting key 990e66cc766cebf644f28c7a1da61b2f
I20260807 08:47:39.320801 6460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:39.312460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.131:59913 (local address 127.4.159.190:43663)
0807 08:47:39.312519 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:39.312521 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:39.312713 (+ 192us) server_negotiation.cc:407] Connection header received
0807 08:47:39.313331 (+ 618us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:39.313332 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:39.313345 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:39.313423 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:47:39.314037 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:39.314627 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:39.315592 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:39.315792 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:39.315848 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:39.318023 (+ 2175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:47:39.318026 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:47:39.318026 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:47:39.318034 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:47:39.319447 (+ 1413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:39.319916 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:39.319918 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:39.319918 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:39.319982 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:39.320186 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:39.320186 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:39.320187 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:39.320336 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:47:39.320403 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:39.320631 (+ 228us) server_negotiation.cc:299] Negotiation successful
0807 08:47:39.320712 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260807 08:47:39.320948 6940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:39.312416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:43663 (local address 127.4.159.131:59913)
0807 08:47:39.312588 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0807 08:47:39.312626 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 08:47:39.313203 (+ 577us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:47:39.313501 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:47:39.313505 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:47:39.313756 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:47:39.313943 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:39.313948 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:39.314706 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:39.314707 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:39.315506 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:39.315509 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:39.315865 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:39.315866 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:39.315989 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:39.316414 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:47:39.316422 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:47:39.317934 (+ 1512us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:47:39.319543 (+ 1609us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:47:39.319548 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:47:39.319550 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:47:39.319843 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:47:39.320040 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:47:39.320041 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:47:39.320041 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:47:39.320131 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:47:39.320394 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:47:39.320399 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:47:39.320506 (+ 107us) client_negotiation.cc:769] Sending connection context
0807 08:47:39.320602 (+ 96us) client_negotiation.cc:240] Negotiation successful
0807 08:47:39.320747 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":86,"threads_started":1}
I20260807 08:47:39.321426 7091 heartbeater.cc:347] Connected to a master server at 127.4.159.190:43663
I20260807 08:47:39.321514 7091 heartbeater.cc:464] Registering TS with master...
I20260807 08:47:39.321682 7091 heartbeater.cc:511] Master 127.4.159.190:43663 requested a full tablet report, sending...
I20260807 08:47:39.322055 6388 ts_manager.cc:194] Registered new tserver with Master: 2348d0b1f3294f8a90178c62af62f40b (127.4.159.131:35523)
I20260807 08:47:39.322530 6388 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.159.131@KRBTEST.COM'} at 127.4.159.131:59913
I20260807 08:47:39.327024 6460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:39.322376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.131:33351 (local address 127.4.159.190:43663)
0807 08:47:39.322414 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:39.322416 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:39.322542 (+ 126us) server_negotiation.cc:407] Connection header received
0807 08:47:39.322652 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:39.322653 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:39.322663 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:39.322689 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:47:39.323104 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:39.323562 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:39.324249 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:39.324428 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:39.324471 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:39.325199 (+ 728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:47:39.325200 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:47:39.325200 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:47:39.325204 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:47:39.326233 (+ 1029us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:39.326534 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:39.326534 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:39.326535 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:39.326579 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:39.326705 (+ 126us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:39.326706 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:39.326706 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:39.326783 (+ 77us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:47:39.326826 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:39.326918 (+ 92us) server_negotiation.cc:299] Negotiation successful
0807 08:47:39.326969 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 08:47:39.327039 7098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:39.322288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:43663 (local address 127.4.159.131:33351)
0807 08:47:39.322485 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0807 08:47:39.322495 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 08:47:39.322556 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:47:39.322737 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:47:39.322738 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:47:39.322930 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:47:39.322991 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:39.322993 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:39.323645 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:39.323646 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:39.324197 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:39.324200 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:39.324476 (+ 276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:39.324477 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:39.324527 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:39.324882 (+ 355us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:47:39.324887 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:47:39.325137 (+ 250us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:47:39.326293 (+ 1156us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:47:39.326293 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:47:39.326294 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:47:39.326479 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:47:39.326624 (+ 145us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:47:39.326624 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:47:39.326625 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:47:39.326672 (+ 47us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:47:39.326825 (+ 153us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:47:39.326826 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:47:39.326858 (+ 32us) client_negotiation.cc:769] Sending connection context
0807 08:47:39.326902 (+ 44us) client_negotiation.cc:240] Negotiation successful
0807 08:47:39.326971 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":145,"threads_started":1}
I20260807 08:47:39.332058 4734 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 08:47:39 dist-test-slave-6zbq krb5kdc[6315](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092459, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 08:47:39 dist-test-slave-6zbq krb5kdc[6315](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092459, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.4.159.190@KRBTEST.COM
I20260807 08:47:39.352823 6460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:39.347178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35644 (local address 127.4.159.190:43663)
0807 08:47:39.347229 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:39.347231 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:39.347238 (+ 7us) server_negotiation.cc:407] Connection header received
0807 08:47:39.347257 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:39.347257 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:39.347265 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:39.347314 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:47:39.347786 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:39.348224 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:39.348886 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:39.349048 (+ 162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:39.349086 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:39.350959 (+ 1873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:47:39.350960 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:47:39.350961 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:47:39.350967 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:47:39.352086 (+ 1119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:39.352337 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:39.352338 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:39.352338 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:39.352380 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:39.352495 (+ 115us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:39.352496 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:39.352496 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:39.352577 (+ 81us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:47:39.352643 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:39.352708 (+ 65us) server_negotiation.cc:299] Negotiation successful
0807 08:47:39.352758 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260807 08:47:39.354192 6388 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35644:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 08:47:39.355751 6388 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:47:39.361619 7109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:39.357639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.129:44539 (local address 127.0.0.1:44724)
0807 08:47:39.357950 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0807 08:47:39.357957 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0807 08:47:39.357984 (+ 27us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:47:39.358285 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:47:39.358292 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:47:39.358293 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:47:39.358374 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:39.358375 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:39.359299 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:39.359300 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:39.360425 (+ 1125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:39.360428 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:39.361337 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:39.361338 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:39.361417 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:39.361430 (+ 13us) client_negotiation.cc:769] Sending connection context
0807 08:47:39.361500 (+ 70us) client_negotiation.cc:240] Negotiation successful
0807 08:47:39.361550 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"mutex_wait_us":2,"spinlock_wait_cycles":384,"thread_start_us":62,"threads_started":1}
I20260807 08:47:39.361675 7110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:39.357753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44724 (local address 127.4.159.129:44539)
0807 08:47:39.358143 (+ 390us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:39.358148 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:39.358157 (+ 9us) server_negotiation.cc:407] Connection header received
0807 08:47:39.358196 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:39.358198 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:39.358213 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:39.358280 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:47:39.358451 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:39.359216 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:39.360498 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:39.361250 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:39.361327 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:39.361389 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:39.361526 (+ 137us) server_negotiation.cc:299] Negotiation successful
0807 08:47:39.361573 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":99,"threads_started":1}
I20260807 08:47:39.361795 7108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:39.357639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.130:45177 (local address 127.0.0.1:46088)
0807 08:47:39.357757 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0807 08:47:39.357797 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 08:47:39.357865 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:47:39.358286 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:47:39.358288 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:47:39.358291 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:47:39.358400 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:39.358403 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:39.359686 (+ 1283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:39.359688 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:39.360709 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:39.360711 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:39.361590 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:39.361591 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:39.361661 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:39.361665 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 08:47:39.361693 (+ 28us) client_negotiation.cc:240] Negotiation successful
0807 08:47:39.361732 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":67,"threads_started":1}
I20260807 08:47:39.362061 7113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:39.357952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46088 (local address 127.4.159.130:45177)
0807 08:47:39.358111 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:39.358114 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:39.358124 (+ 10us) server_negotiation.cc:407] Connection header received
0807 08:47:39.358160 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:39.358162 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:39.358177 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:39.358481 (+ 304us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:47:39.358542 (+ 61us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:39.359607 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:39.360782 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:39.361496 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:39.361855 (+ 359us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:39.361904 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:39.361930 (+ 26us) server_negotiation.cc:299] Negotiation successful
0807 08:47:39.361966 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":80,"threads_started":1}
I20260807 08:47:39.362632 6792 tablet_service.cc:1467] Processing CreateTablet for tablet b5e8b2e921d04bc196ede8a6f4a763ac (DEFAULT_TABLE table=test-table [id=0b562af71aaf46d39aa5d2c04fa4a002]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:47:39.362636 6584 tablet_service.cc:1467] Processing CreateTablet for tablet b5e8b2e921d04bc196ede8a6f4a763ac (DEFAULT_TABLE table=test-table [id=0b562af71aaf46d39aa5d2c04fa4a002]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:47:39.362993 6792 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b5e8b2e921d04bc196ede8a6f4a763ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:47:39.362993 6584 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b5e8b2e921d04bc196ede8a6f4a763ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:47:39.365705 7114 tablet_bootstrap.cc:492] T b5e8b2e921d04bc196ede8a6f4a763ac P 661ee1d16b2648d98bce6027b5132089: Bootstrap starting.
I20260807 08:47:39.365631 7116 tablet_bootstrap.cc:492] T b5e8b2e921d04bc196ede8a6f4a763ac P 48b5111c255f4226b5ce348cc76a2d1b: Bootstrap starting.
I20260807 08:47:39.366210 7111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:39.357839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.131:35523 (local address 127.0.0.1:55818)
0807 08:47:39.358113 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0807 08:47:39.358123 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 08:47:39.358183 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:47:39.358425 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:47:39.358426 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:47:39.358428 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:47:39.358458 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:39.358460 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:39.360773 (+ 2313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:39.360773 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:39.361501 (+ 728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:39.361503 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:39.365958 (+ 4455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:39.365959 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:39.366024 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:39.366029 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 08:47:39.366106 (+ 77us) client_negotiation.cc:240] Negotiation successful
0807 08:47:39.366148 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":48,"threads_started":1}
I20260807 08:47:39.366460 7116 tablet_bootstrap.cc:654] T b5e8b2e921d04bc196ede8a6f4a763ac P 48b5111c255f4226b5ce348cc76a2d1b: Neither blocks nor log segments found. Creating new log.
I20260807 08:47:39.366478 7114 tablet_bootstrap.cc:654] T b5e8b2e921d04bc196ede8a6f4a763ac P 661ee1d16b2648d98bce6027b5132089: Neither blocks nor log segments found. Creating new log.
I20260807 08:47:39.366514 7112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:39.357939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55818 (local address 127.4.159.131:35523)
0807 08:47:39.358109 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:39.358113 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:39.358172 (+ 59us) server_negotiation.cc:407] Connection header received
0807 08:47:39.358239 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:39.358241 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:39.358254 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:39.358320 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:47:39.358543 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:39.359601 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:39.361579 (+ 1978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:39.365811 (+ 4232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:39.365875 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:39.365925 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:39.366142 (+ 217us) server_negotiation.cc:299] Negotiation successful
0807 08:47:39.366184 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":86,"threads_started":1}
I20260807 08:47:39.366748 7116 log.cc:824] T b5e8b2e921d04bc196ede8a6f4a763ac P 48b5111c255f4226b5ce348cc76a2d1b: Log is configured to *not* fsync() on all Append() calls
I20260807 08:47:39.366959 7114 log.cc:824] T b5e8b2e921d04bc196ede8a6f4a763ac P 661ee1d16b2648d98bce6027b5132089: Log is configured to *not* fsync() on all Append() calls
I20260807 08:47:39.367066 6996 tablet_service.cc:1467] Processing CreateTablet for tablet b5e8b2e921d04bc196ede8a6f4a763ac (DEFAULT_TABLE table=test-table [id=0b562af71aaf46d39aa5d2c04fa4a002]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:47:39.367363 6996 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b5e8b2e921d04bc196ede8a6f4a763ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:47:39.367487 7116 tablet_bootstrap.cc:492] T b5e8b2e921d04bc196ede8a6f4a763ac P 48b5111c255f4226b5ce348cc76a2d1b: No bootstrap required, opened a new log
I20260807 08:47:39.367559 7116 ts_tablet_manager.cc:1397] T b5e8b2e921d04bc196ede8a6f4a763ac P 48b5111c255f4226b5ce348cc76a2d1b: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260807 08:47:39.367683 7114 tablet_bootstrap.cc:492] T b5e8b2e921d04bc196ede8a6f4a763ac P 661ee1d16b2648d98bce6027b5132089: No bootstrap required, opened a new log
I20260807 08:47:39.367745 7114 ts_tablet_manager.cc:1397] T b5e8b2e921d04bc196ede8a6f4a763ac P 661ee1d16b2648d98bce6027b5132089: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260807 08:47:39.369496 7116 raft_consensus.cc:348] T b5e8b2e921d04bc196ede8a6f4a763ac P 48b5111c255f4226b5ce348cc76a2d1b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b5111c255f4226b5ce348cc76a2d1b" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 44539 } } peers { permanent_uuid: "2348d0b1f3294f8a90178c62af62f40b" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 35523 } } peers { permanent_uuid: "661ee1d16b2648d98bce6027b5132089" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 45177 } }
I20260807 08:47:39.369585 7114 raft_consensus.cc:348] T b5e8b2e921d04bc196ede8a6f4a763ac P 661ee1d16b2648d98bce6027b5132089 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b5111c255f4226b5ce348cc76a2d1b" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 44539 } } peers { permanent_uuid: "2348d0b1f3294f8a90178c62af62f40b" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 35523 } } peers { permanent_uuid: "661ee1d16b2648d98bce6027b5132089" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 45177 } }
I20260807 08:47:39.369668 7116 raft_consensus.cc:374] T b5e8b2e921d04bc196ede8a6f4a763ac P 48b5111c255f4226b5ce348cc76a2d1b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:47:39.369712 7116 raft_consensus.cc:729] T b5e8b2e921d04bc196ede8a6f4a763ac P 48b5111c255f4226b5ce348cc76a2d1b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48b5111c255f4226b5ce348cc76a2d1b, State: Initialized, Role: FOLLOWER
I20260807 08:47:39.369714 7114 raft_consensus.cc:374] T b5e8b2e921d04bc196ede8a6f4a763ac P 661ee1d16b2648d98bce6027b5132089 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:47:39.369753 7114 raft_consensus.cc:729] T b5e8b2e921d04bc196ede8a6f4a763ac P 661ee1d16b2648d98bce6027b5132089 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 661ee1d16b2648d98bce6027b5132089, State: Initialized, Role: FOLLOWER
I20260807 08:47:39.369865 7114 consensus_queue.cc:260] T b5e8b2e921d04bc196ede8a6f4a763ac P 661ee1d16b2648d98bce6027b5132089 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b5111c255f4226b5ce348cc76a2d1b" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 44539 } } peers { permanent_uuid: "2348d0b1f3294f8a90178c62af62f40b" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 35523 } } peers { permanent_uuid: "661ee1d16b2648d98bce6027b5132089" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 45177 } }
I20260807 08:47:39.369865 7116 consensus_queue.cc:260] T b5e8b2e921d04bc196ede8a6f4a763ac P 48b5111c255f4226b5ce348cc76a2d1b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b5111c255f4226b5ce348cc76a2d1b" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 44539 } } peers { permanent_uuid: "2348d0b1f3294f8a90178c62af62f40b" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 35523 } } peers { permanent_uuid: "661ee1d16b2648d98bce6027b5132089" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 45177 } }
I20260807 08:47:39.369951 7120 tablet_bootstrap.cc:492] T b5e8b2e921d04bc196ede8a6f4a763ac P 2348d0b1f3294f8a90178c62af62f40b: Bootstrap starting.
I20260807 08:47:39.370132 7116 ts_tablet_manager.cc:1428] T b5e8b2e921d04bc196ede8a6f4a763ac P 48b5111c255f4226b5ce348cc76a2d1b: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260807 08:47:39.370132 7114 ts_tablet_manager.cc:1428] T b5e8b2e921d04bc196ede8a6f4a763ac P 661ee1d16b2648d98bce6027b5132089: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260807 08:47:39.370265 6687 heartbeater.cc:503] Master 127.4.159.190:43663 was elected leader, sending a full tablet report...
I20260807 08:47:39.370405 6886 heartbeater.cc:503] Master 127.4.159.190:43663 was elected leader, sending a full tablet report...
I20260807 08:47:39.370810 7120 tablet_bootstrap.cc:654] T b5e8b2e921d04bc196ede8a6f4a763ac P 2348d0b1f3294f8a90178c62af62f40b: Neither blocks nor log segments found. Creating new log.
I20260807 08:47:39.371114 7120 log.cc:824] T b5e8b2e921d04bc196ede8a6f4a763ac P 2348d0b1f3294f8a90178c62af62f40b: Log is configured to *not* fsync() on all Append() calls
I20260807 08:47:39.371764 7120 tablet_bootstrap.cc:492] T b5e8b2e921d04bc196ede8a6f4a763ac P 2348d0b1f3294f8a90178c62af62f40b: No bootstrap required, opened a new log
I20260807 08:47:39.371827 7120 ts_tablet_manager.cc:1397] T b5e8b2e921d04bc196ede8a6f4a763ac P 2348d0b1f3294f8a90178c62af62f40b: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260807 08:47:39.373077 7120 raft_consensus.cc:348] T b5e8b2e921d04bc196ede8a6f4a763ac P 2348d0b1f3294f8a90178c62af62f40b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b5111c255f4226b5ce348cc76a2d1b" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 44539 } } peers { permanent_uuid: "2348d0b1f3294f8a90178c62af62f40b" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 35523 } } peers { permanent_uuid: "661ee1d16b2648d98bce6027b5132089" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 45177 } }
I20260807 08:47:39.373225 7120 raft_consensus.cc:374] T b5e8b2e921d04bc196ede8a6f4a763ac P 2348d0b1f3294f8a90178c62af62f40b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:47:39.373270 7120 raft_consensus.cc:729] T b5e8b2e921d04bc196ede8a6f4a763ac P 2348d0b1f3294f8a90178c62af62f40b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2348d0b1f3294f8a90178c62af62f40b, State: Initialized, Role: FOLLOWER
I20260807 08:47:39.373409 7120 consensus_queue.cc:260] T b5e8b2e921d04bc196ede8a6f4a763ac P 2348d0b1f3294f8a90178c62af62f40b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b5111c255f4226b5ce348cc76a2d1b" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 44539 } } peers { permanent_uuid: "2348d0b1f3294f8a90178c62af62f40b" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 35523 } } peers { permanent_uuid: "661ee1d16b2648d98bce6027b5132089" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 45177 } }
I20260807 08:47:39.373651 7120 ts_tablet_manager.cc:1428] T b5e8b2e921d04bc196ede8a6f4a763ac P 2348d0b1f3294f8a90178c62af62f40b: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260807 08:47:39.373705 7091 heartbeater.cc:503] Master 127.4.159.190:43663 was elected leader, sending a full tablet report...
W20260807 08:47:39.425064 6887 tablet.cc:2369] T b5e8b2e921d04bc196ede8a6f4a763ac P 661ee1d16b2648d98bce6027b5132089: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 08:47:39.520895 6688 tablet.cc:2369] T b5e8b2e921d04bc196ede8a6f4a763ac P 48b5111c255f4226b5ce348cc76a2d1b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 08:47:39.538017 7125 raft_consensus.cc:482] T b5e8b2e921d04bc196ede8a6f4a763ac P 2348d0b1f3294f8a90178c62af62f40b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:47:39.538101 7125 raft_consensus.cc:504] T b5e8b2e921d04bc196ede8a6f4a763ac P 2348d0b1f3294f8a90178c62af62f40b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b5111c255f4226b5ce348cc76a2d1b" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 44539 } } peers { permanent_uuid: "2348d0b1f3294f8a90178c62af62f40b" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 35523 } } peers { permanent_uuid: "661ee1d16b2648d98bce6027b5132089" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 45177 } }
I20260807 08:47:39.538431 7125 leader_election.cc:288] T b5e8b2e921d04bc196ede8a6f4a763ac P 2348d0b1f3294f8a90178c62af62f40b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48b5111c255f4226b5ce348cc76a2d1b (127.4.159.129:44539), 661ee1d16b2648d98bce6027b5132089 (127.4.159.130:45177)
I20260807 08:47:39.541963 7113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:39.538616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.131:60987 (local address 127.4.159.130:45177)
0807 08:47:39.538659 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:39.538661 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:39.538785 (+ 124us) server_negotiation.cc:407] Connection header received
0807 08:47:39.538818 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:39.538819 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:39.538828 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:39.538870 (+ 42us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:47:39.538987 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:39.539665 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:39.541070 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:39.541632 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:39.541694 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:39.541702 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:39.541860 (+ 158us) server_negotiation.cc:299] Negotiation successful
0807 08:47:39.541898 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 08:47:39.541965 7110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:39.538615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.131:44109 (local address 127.4.159.129:44539)
0807 08:47:39.538654 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:39.538656 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:39.538664 (+ 8us) server_negotiation.cc:407] Connection header received
0807 08:47:39.538723 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:39.538724 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:39.538733 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:39.538782 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:47:39.538935 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:39.539665 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:39.541070 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:39.541618 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:39.541694 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:39.541702 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:39.541860 (+ 158us) server_negotiation.cc:299] Negotiation successful
0807 08:47:39.541907 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 08:47:39.541978 6940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:39.538548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.129:44539 (local address 127.4.159.131:44109)
0807 08:47:39.538620 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0807 08:47:39.538631 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 08:47:39.538673 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:47:39.538803 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:47:39.538804 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:47:39.538806 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:47:39.538871 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:39.538872 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:39.539732 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:39.539733 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:39.540997 (+ 1264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:39.541000 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:39.541691 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:39.541692 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:39.541799 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:39.541802 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 08:47:39.541849 (+ 47us) client_negotiation.cc:240] Negotiation successful
0807 08:47:39.541891 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53,"mutex_wait_us":53}
I20260807 08:47:39.541992 7126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:39.538580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.130:45177 (local address 127.4.159.131:60987)
0807 08:47:39.538708 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0807 08:47:39.538717 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 08:47:39.538777 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:47:39.538871 (+ 94us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:47:39.538871 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:47:39.538873 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:47:39.538932 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:39.538933 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:39.539732 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:39.539733 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:39.540997 (+ 1264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:39.541000 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:39.541691 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:39.541692 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:39.541791 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:39.541795 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 08:47:39.541849 (+ 54us) client_negotiation.cc:240] Negotiation successful
0807 08:47:39.541891 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"mutex_wait_us":55,"thread_start_us":78,"threads_started":1}
I20260807 08:47:39.542376 6614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5e8b2e921d04bc196ede8a6f4a763ac" candidate_uuid: "2348d0b1f3294f8a90178c62af62f40b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48b5111c255f4226b5ce348cc76a2d1b" is_pre_election: true
I20260807 08:47:39.542475 6817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5e8b2e921d04bc196ede8a6f4a763ac" candidate_uuid: "2348d0b1f3294f8a90178c62af62f40b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "661ee1d16b2648d98bce6027b5132089" is_pre_election: true
I20260807 08:47:39.542510 6614 raft_consensus.cc:2454] T b5e8b2e921d04bc196ede8a6f4a763ac P 48b5111c255f4226b5ce348cc76a2d1b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2348d0b1f3294f8a90178c62af62f40b in term 0.
I20260807 08:47:39.542608 6817 raft_consensus.cc:2454] T b5e8b2e921d04bc196ede8a6f4a763ac P 661ee1d16b2648d98bce6027b5132089 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2348d0b1f3294f8a90178c62af62f40b in term 0.
I20260807 08:47:39.542758 6928 leader_election.cc:302] T b5e8b2e921d04bc196ede8a6f4a763ac P 2348d0b1f3294f8a90178c62af62f40b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2348d0b1f3294f8a90178c62af62f40b, 48b5111c255f4226b5ce348cc76a2d1b; no voters:
I20260807 08:47:39.542893 7125 raft_consensus.cc:2781] T b5e8b2e921d04bc196ede8a6f4a763ac P 2348d0b1f3294f8a90178c62af62f40b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:47:39.542950 7125 raft_consensus.cc:482] T b5e8b2e921d04bc196ede8a6f4a763ac P 2348d0b1f3294f8a90178c62af62f40b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:47:39.542973 7125 raft_consensus.cc:3037] T b5e8b2e921d04bc196ede8a6f4a763ac P 2348d0b1f3294f8a90178c62af62f40b [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:47:39.543830 7125 raft_consensus.cc:504] T b5e8b2e921d04bc196ede8a6f4a763ac P 2348d0b1f3294f8a90178c62af62f40b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b5111c255f4226b5ce348cc76a2d1b" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 44539 } } peers { permanent_uuid: "2348d0b1f3294f8a90178c62af62f40b" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 35523 } } peers { permanent_uuid: "661ee1d16b2648d98bce6027b5132089" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 45177 } }
I20260807 08:47:39.543946 7125 leader_election.cc:288] T b5e8b2e921d04bc196ede8a6f4a763ac P 2348d0b1f3294f8a90178c62af62f40b [CANDIDATE]: Term 1 election: Requested vote from peers 48b5111c255f4226b5ce348cc76a2d1b (127.4.159.129:44539), 661ee1d16b2648d98bce6027b5132089 (127.4.159.130:45177)
I20260807 08:47:39.544090 6817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5e8b2e921d04bc196ede8a6f4a763ac" candidate_uuid: "2348d0b1f3294f8a90178c62af62f40b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "661ee1d16b2648d98bce6027b5132089"
I20260807 08:47:39.544158 6817 raft_consensus.cc:3037] T b5e8b2e921d04bc196ede8a6f4a763ac P 661ee1d16b2648d98bce6027b5132089 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:47:39.544240 6614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5e8b2e921d04bc196ede8a6f4a763ac" candidate_uuid: "2348d0b1f3294f8a90178c62af62f40b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48b5111c255f4226b5ce348cc76a2d1b"
I20260807 08:47:39.544312 6614 raft_consensus.cc:3037] T b5e8b2e921d04bc196ede8a6f4a763ac P 48b5111c255f4226b5ce348cc76a2d1b [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:47:39.545020 6817 raft_consensus.cc:2454] T b5e8b2e921d04bc196ede8a6f4a763ac P 661ee1d16b2648d98bce6027b5132089 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2348d0b1f3294f8a90178c62af62f40b in term 1.
I20260807 08:47:39.545064 6614 raft_consensus.cc:2454] T b5e8b2e921d04bc196ede8a6f4a763ac P 48b5111c255f4226b5ce348cc76a2d1b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2348d0b1f3294f8a90178c62af62f40b in term 1.
I20260807 08:47:39.545190 6929 leader_election.cc:302] T b5e8b2e921d04bc196ede8a6f4a763ac P 2348d0b1f3294f8a90178c62af62f40b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2348d0b1f3294f8a90178c62af62f40b, 661ee1d16b2648d98bce6027b5132089; no voters:
I20260807 08:47:39.545315 7125 raft_consensus.cc:2781] T b5e8b2e921d04bc196ede8a6f4a763ac P 2348d0b1f3294f8a90178c62af62f40b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:47:39.545491 7125 raft_consensus.cc:686] T b5e8b2e921d04bc196ede8a6f4a763ac P 2348d0b1f3294f8a90178c62af62f40b [term 1 LEADER]: Becoming Leader. State: Replica: 2348d0b1f3294f8a90178c62af62f40b, State: Running, Role: LEADER
I20260807 08:47:39.545574 7125 consensus_queue.cc:237] T b5e8b2e921d04bc196ede8a6f4a763ac P 2348d0b1f3294f8a90178c62af62f40b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b5111c255f4226b5ce348cc76a2d1b" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 44539 } } peers { permanent_uuid: "2348d0b1f3294f8a90178c62af62f40b" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 35523 } } peers { permanent_uuid: "661ee1d16b2648d98bce6027b5132089" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 45177 } }
I20260807 08:47:39.546411 6388 catalog_manager.cc:5495] T b5e8b2e921d04bc196ede8a6f4a763ac P 2348d0b1f3294f8a90178c62af62f40b reported cstate change: term changed from 0 to 1, leader changed from <none> to 2348d0b1f3294f8a90178c62af62f40b (127.4.159.131). New cstate: current_term: 1 leader_uuid: "2348d0b1f3294f8a90178c62af62f40b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b5111c255f4226b5ce348cc76a2d1b" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 44539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2348d0b1f3294f8a90178c62af62f40b" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 35523 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "661ee1d16b2648d98bce6027b5132089" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 45177 } health_report { overall_health: UNKNOWN } } }
W20260807 08:47:39.569860 7092 tablet.cc:2369] T b5e8b2e921d04bc196ede8a6f4a763ac P 2348d0b1f3294f8a90178c62af62f40b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 08:47:39.585465 7112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:39.583448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55828 (local address 127.4.159.131:35523)
0807 08:47:39.583491 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:39.583492 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:39.583518 (+ 26us) server_negotiation.cc:407] Connection header received
0807 08:47:39.583589 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:39.583590 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:39.583604 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:39.583656 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:47:39.583763 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:39.584189 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:39.584875 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:39.585043 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:39.585086 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:39.585223 (+ 137us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:47:39.585301 (+ 78us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:47:39.585355 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:39.585388 (+ 33us) server_negotiation.cc:299] Negotiation successful
0807 08:47:39.585403 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 08:47:39.586946 6614 raft_consensus.cc:1261] T b5e8b2e921d04bc196ede8a6f4a763ac P 48b5111c255f4226b5ce348cc76a2d1b [term 1 FOLLOWER]: Refusing update from remote peer 2348d0b1f3294f8a90178c62af62f40b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:47:39.587217 7127 consensus_queue.cc:1035] T b5e8b2e921d04bc196ede8a6f4a763ac P 2348d0b1f3294f8a90178c62af62f40b [LEADER]: Connected to new peer: Peer: permanent_uuid: "48b5111c255f4226b5ce348cc76a2d1b" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 44539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:47:39.588718 6817 raft_consensus.cc:1261] T b5e8b2e921d04bc196ede8a6f4a763ac P 661ee1d16b2648d98bce6027b5132089 [term 1 FOLLOWER]: Refusing update from remote peer 2348d0b1f3294f8a90178c62af62f40b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:47:39.589026 7125 consensus_queue.cc:1035] T b5e8b2e921d04bc196ede8a6f4a763ac P 2348d0b1f3294f8a90178c62af62f40b [LEADER]: Connected to new peer: Peer: permanent_uuid: "661ee1d16b2648d98bce6027b5132089" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 45177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:47:39.592958 6388 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35644:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:47:39.593111 6388 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35644:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:47:39.593878 6388 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a9b26c4a4f35408cac8649134220bab5: Sending DeleteTablet for 3 replicas of tablet b5e8b2e921d04bc196ede8a6f4a763ac
I20260807 08:47:39.594375 6584 tablet_service.cc:1514] Processing DeleteTablet for tablet b5e8b2e921d04bc196ede8a6f4a763ac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:47:39 UTC) from {username='kudu', principal='kudu/127.4.159.190@KRBTEST.COM'} at 127.0.0.1:44724
I20260807 08:47:39.594398 6996 tablet_service.cc:1514] Processing DeleteTablet for tablet b5e8b2e921d04bc196ede8a6f4a763ac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:47:39 UTC) from {username='kudu', principal='kudu/127.4.159.190@KRBTEST.COM'} at 127.0.0.1:55818
I20260807 08:47:39.594398 6792 tablet_service.cc:1514] Processing DeleteTablet for tablet b5e8b2e921d04bc196ede8a6f4a763ac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:47:39 UTC) from {username='kudu', principal='kudu/127.4.159.190@KRBTEST.COM'} at 127.0.0.1:46088
I20260807 08:47:39.594614 7145 tablet_replica.cc:330] T b5e8b2e921d04bc196ede8a6f4a763ac P 661ee1d16b2648d98bce6027b5132089: stopping tablet replica
I20260807 08:47:39.594642 7142 tablet_replica.cc:330] T b5e8b2e921d04bc196ede8a6f4a763ac P 48b5111c255f4226b5ce348cc76a2d1b: stopping tablet replica
I20260807 08:47:39.594714 7145 raft_consensus.cc:2229] T b5e8b2e921d04bc196ede8a6f4a763ac P 661ee1d16b2648d98bce6027b5132089 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:47:39.594781 7142 raft_consensus.cc:2229] T b5e8b2e921d04bc196ede8a6f4a763ac P 48b5111c255f4226b5ce348cc76a2d1b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:47:39.594899 7142 raft_consensus.cc:2258] T b5e8b2e921d04bc196ede8a6f4a763ac P 48b5111c255f4226b5ce348cc76a2d1b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:47:39.594895 6388 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35644:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 08:47:39.595028 6388 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:47:39.595073 7145 raft_consensus.cc:2258] T b5e8b2e921d04bc196ede8a6f4a763ac P 661ee1d16b2648d98bce6027b5132089 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:47:39.595429 7144 tablet_replica.cc:330] T b5e8b2e921d04bc196ede8a6f4a763ac P 2348d0b1f3294f8a90178c62af62f40b: stopping tablet replica
I20260807 08:47:39.595513 7144 raft_consensus.cc:2229] T b5e8b2e921d04bc196ede8a6f4a763ac P 2348d0b1f3294f8a90178c62af62f40b [term 1 LEADER]: Raft consensus shutting down.
I20260807 08:47:39.595624 7144 raft_consensus.cc:2258] T b5e8b2e921d04bc196ede8a6f4a763ac P 2348d0b1f3294f8a90178c62af62f40b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:47:39.595760 7145 ts_tablet_manager.cc:1905] T b5e8b2e921d04bc196ede8a6f4a763ac P 661ee1d16b2648d98bce6027b5132089: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:47:39.596048 7144 ts_tablet_manager.cc:1905] T b5e8b2e921d04bc196ede8a6f4a763ac P 2348d0b1f3294f8a90178c62af62f40b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:47:39.596382 6792 tablet_service.cc:1467] Processing CreateTablet for tablet 40c8edc51b414cb0ac6aabe17cd5026f (DEFAULT_TABLE table=test-table [id=54a2778ed6f4448892dfb2dd31d87536]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:47:39.596493 6792 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40c8edc51b414cb0ac6aabe17cd5026f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:47:39.597440 7142 ts_tablet_manager.cc:1905] T b5e8b2e921d04bc196ede8a6f4a763ac P 48b5111c255f4226b5ce348cc76a2d1b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:47:39.597627 7114 tablet_bootstrap.cc:492] T 40c8edc51b414cb0ac6aabe17cd5026f P 661ee1d16b2648d98bce6027b5132089: Bootstrap starting.
I20260807 08:47:39.597633 7144 ts_tablet_manager.cc:1918] T b5e8b2e921d04bc196ede8a6f4a763ac P 2348d0b1f3294f8a90178c62af62f40b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:47:39.597695 7144 log.cc:1196] T b5e8b2e921d04bc196ede8a6f4a763ac P 2348d0b1f3294f8a90178c62af62f40b: Deleting WAL directory at /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786092453486433-4734-0/minicluster-data/ts-2/wal/wals/b5e8b2e921d04bc196ede8a6f4a763ac
I20260807 08:47:39.597944 7144 ts_tablet_manager.cc:1939] T b5e8b2e921d04bc196ede8a6f4a763ac P 2348d0b1f3294f8a90178c62af62f40b: Deleting consensus metadata
I20260807 08:47:39.597999 7114 tablet_bootstrap.cc:654] T 40c8edc51b414cb0ac6aabe17cd5026f P 661ee1d16b2648d98bce6027b5132089: Neither blocks nor log segments found. Creating new log.
I20260807 08:47:39.598726 6362 catalog_manager.cc:4841] TS 2348d0b1f3294f8a90178c62af62f40b (127.4.159.131:35523): tablet b5e8b2e921d04bc196ede8a6f4a763ac (table test-table [id=0b562af71aaf46d39aa5d2c04fa4a002]) successfully deleted
I20260807 08:47:39.598807 6584 tablet_service.cc:1467] Processing CreateTablet for tablet 40c8edc51b414cb0ac6aabe17cd5026f (DEFAULT_TABLE table=test-table [id=54a2778ed6f4448892dfb2dd31d87536]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:47:39.598953 6584 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40c8edc51b414cb0ac6aabe17cd5026f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:47:39.598959 7142 ts_tablet_manager.cc:1918] T b5e8b2e921d04bc196ede8a6f4a763ac P 48b5111c255f4226b5ce348cc76a2d1b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:47:39.599056 7142 log.cc:1196] T b5e8b2e921d04bc196ede8a6f4a763ac P 48b5111c255f4226b5ce348cc76a2d1b: Deleting WAL directory at /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786092453486433-4734-0/minicluster-data/ts-0/wal/wals/b5e8b2e921d04bc196ede8a6f4a763ac
I20260807 08:47:39.599283 7142 ts_tablet_manager.cc:1939] T b5e8b2e921d04bc196ede8a6f4a763ac P 48b5111c255f4226b5ce348cc76a2d1b: Deleting consensus metadata
I20260807 08:47:39.599733 7114 tablet_bootstrap.cc:492] T 40c8edc51b414cb0ac6aabe17cd5026f P 661ee1d16b2648d98bce6027b5132089: No bootstrap required, opened a new log
I20260807 08:47:39.599774 7114 ts_tablet_manager.cc:1397] T 40c8edc51b414cb0ac6aabe17cd5026f P 661ee1d16b2648d98bce6027b5132089: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260807 08:47:39.599903 7114 raft_consensus.cc:348] T 40c8edc51b414cb0ac6aabe17cd5026f P 661ee1d16b2648d98bce6027b5132089 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b5111c255f4226b5ce348cc76a2d1b" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 44539 } } peers { permanent_uuid: "661ee1d16b2648d98bce6027b5132089" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 45177 } } peers { permanent_uuid: "2348d0b1f3294f8a90178c62af62f40b" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 35523 } }
I20260807 08:47:39.599953 7114 raft_consensus.cc:374] T 40c8edc51b414cb0ac6aabe17cd5026f P 661ee1d16b2648d98bce6027b5132089 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:47:39.599972 7114 raft_consensus.cc:729] T 40c8edc51b414cb0ac6aabe17cd5026f P 661ee1d16b2648d98bce6027b5132089 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 661ee1d16b2648d98bce6027b5132089, State: Initialized, Role: FOLLOWER
I20260807 08:47:39.600008 7114 consensus_queue.cc:260] T 40c8edc51b414cb0ac6aabe17cd5026f P 661ee1d16b2648d98bce6027b5132089 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b5111c255f4226b5ce348cc76a2d1b" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 44539 } } peers { permanent_uuid: "661ee1d16b2648d98bce6027b5132089" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 45177 } } peers { permanent_uuid: "2348d0b1f3294f8a90178c62af62f40b" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 35523 } }
I20260807 08:47:39.600051 7116 tablet_bootstrap.cc:492] T 40c8edc51b414cb0ac6aabe17cd5026f P 48b5111c255f4226b5ce348cc76a2d1b: Bootstrap starting.
I20260807 08:47:39.600092 7114 ts_tablet_manager.cc:1428] T 40c8edc51b414cb0ac6aabe17cd5026f P 661ee1d16b2648d98bce6027b5132089: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:39.600278 7145 ts_tablet_manager.cc:1918] T b5e8b2e921d04bc196ede8a6f4a763ac P 661ee1d16b2648d98bce6027b5132089: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:47:39.600373 7145 log.cc:1196] T b5e8b2e921d04bc196ede8a6f4a763ac P 661ee1d16b2648d98bce6027b5132089: Deleting WAL directory at /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786092453486433-4734-0/minicluster-data/ts-1/wal/wals/b5e8b2e921d04bc196ede8a6f4a763ac
I20260807 08:47:39.600417 7116 tablet_bootstrap.cc:654] T 40c8edc51b414cb0ac6aabe17cd5026f P 48b5111c255f4226b5ce348cc76a2d1b: Neither blocks nor log segments found. Creating new log.
I20260807 08:47:39.600616 7145 ts_tablet_manager.cc:1939] T b5e8b2e921d04bc196ede8a6f4a763ac P 661ee1d16b2648d98bce6027b5132089: Deleting consensus metadata
I20260807 08:47:39.600842 6996 tablet_service.cc:1467] Processing CreateTablet for tablet 40c8edc51b414cb0ac6aabe17cd5026f (DEFAULT_TABLE table=test-table [id=54a2778ed6f4448892dfb2dd31d87536]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:47:39.600946 6996 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40c8edc51b414cb0ac6aabe17cd5026f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:47:39.600993 6364 catalog_manager.cc:4841] TS 661ee1d16b2648d98bce6027b5132089 (127.4.159.130:45177): tablet b5e8b2e921d04bc196ede8a6f4a763ac (table test-table [id=0b562af71aaf46d39aa5d2c04fa4a002]) successfully deleted
I20260807 08:47:39.601325 7116 tablet_bootstrap.cc:492] T 40c8edc51b414cb0ac6aabe17cd5026f P 48b5111c255f4226b5ce348cc76a2d1b: No bootstrap required, opened a new log
I20260807 08:47:39.601372 7116 ts_tablet_manager.cc:1397] T 40c8edc51b414cb0ac6aabe17cd5026f P 48b5111c255f4226b5ce348cc76a2d1b: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260807 08:47:39.601508 7116 raft_consensus.cc:348] T 40c8edc51b414cb0ac6aabe17cd5026f P 48b5111c255f4226b5ce348cc76a2d1b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b5111c255f4226b5ce348cc76a2d1b" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 44539 } } peers { permanent_uuid: "661ee1d16b2648d98bce6027b5132089" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 45177 } } peers { permanent_uuid: "2348d0b1f3294f8a90178c62af62f40b" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 35523 } }
I20260807 08:47:39.601559 7116 raft_consensus.cc:374] T 40c8edc51b414cb0ac6aabe17cd5026f P 48b5111c255f4226b5ce348cc76a2d1b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:47:39.601576 7116 raft_consensus.cc:729] T 40c8edc51b414cb0ac6aabe17cd5026f P 48b5111c255f4226b5ce348cc76a2d1b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48b5111c255f4226b5ce348cc76a2d1b, State: Initialized, Role: FOLLOWER
I20260807 08:47:39.601626 7116 consensus_queue.cc:260] T 40c8edc51b414cb0ac6aabe17cd5026f P 48b5111c255f4226b5ce348cc76a2d1b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b5111c255f4226b5ce348cc76a2d1b" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 44539 } } peers { permanent_uuid: "661ee1d16b2648d98bce6027b5132089" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 45177 } } peers { permanent_uuid: "2348d0b1f3294f8a90178c62af62f40b" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 35523 } }
I20260807 08:47:39.601723 7116 ts_tablet_manager.cc:1428] T 40c8edc51b414cb0ac6aabe17cd5026f P 48b5111c255f4226b5ce348cc76a2d1b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:39.602133 7120 tablet_bootstrap.cc:492] T 40c8edc51b414cb0ac6aabe17cd5026f P 2348d0b1f3294f8a90178c62af62f40b: Bootstrap starting.
I20260807 08:47:39.602238 6362 catalog_manager.cc:4841] TS 48b5111c255f4226b5ce348cc76a2d1b (127.4.159.129:44539): tablet b5e8b2e921d04bc196ede8a6f4a763ac (table test-table [id=0b562af71aaf46d39aa5d2c04fa4a002]) successfully deleted
I20260807 08:47:39.602550 7120 tablet_bootstrap.cc:654] T 40c8edc51b414cb0ac6aabe17cd5026f P 2348d0b1f3294f8a90178c62af62f40b: Neither blocks nor log segments found. Creating new log.
I20260807 08:47:39.603137 7120 tablet_bootstrap.cc:492] T 40c8edc51b414cb0ac6aabe17cd5026f P 2348d0b1f3294f8a90178c62af62f40b: No bootstrap required, opened a new log
I20260807 08:47:39.603178 7120 ts_tablet_manager.cc:1397] T 40c8edc51b414cb0ac6aabe17cd5026f P 2348d0b1f3294f8a90178c62af62f40b: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 08:47:39.603322 7120 raft_consensus.cc:348] T 40c8edc51b414cb0ac6aabe17cd5026f P 2348d0b1f3294f8a90178c62af62f40b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b5111c255f4226b5ce348cc76a2d1b" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 44539 } } peers { permanent_uuid: "661ee1d16b2648d98bce6027b5132089" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 45177 } } peers { permanent_uuid: "2348d0b1f3294f8a90178c62af62f40b" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 35523 } }
I20260807 08:47:39.603391 7120 raft_consensus.cc:374] T 40c8edc51b414cb0ac6aabe17cd5026f P 2348d0b1f3294f8a90178c62af62f40b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:47:39.603420 7120 raft_consensus.cc:729] T 40c8edc51b414cb0ac6aabe17cd5026f P 2348d0b1f3294f8a90178c62af62f40b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2348d0b1f3294f8a90178c62af62f40b, State: Initialized, Role: FOLLOWER
I20260807 08:47:39.603462 7120 consensus_queue.cc:260] T 40c8edc51b414cb0ac6aabe17cd5026f P 2348d0b1f3294f8a90178c62af62f40b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b5111c255f4226b5ce348cc76a2d1b" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 44539 } } peers { permanent_uuid: "661ee1d16b2648d98bce6027b5132089" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 45177 } } peers { permanent_uuid: "2348d0b1f3294f8a90178c62af62f40b" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 35523 } }
I20260807 08:47:39.603541 7120 ts_tablet_manager.cc:1428] T 40c8edc51b414cb0ac6aabe17cd5026f P 2348d0b1f3294f8a90178c62af62f40b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:40.059240 7123 raft_consensus.cc:482] T 40c8edc51b414cb0ac6aabe17cd5026f P 48b5111c255f4226b5ce348cc76a2d1b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:47:40.059324 7123 raft_consensus.cc:504] T 40c8edc51b414cb0ac6aabe17cd5026f P 48b5111c255f4226b5ce348cc76a2d1b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b5111c255f4226b5ce348cc76a2d1b" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 44539 } } peers { permanent_uuid: "661ee1d16b2648d98bce6027b5132089" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 45177 } } peers { permanent_uuid: "2348d0b1f3294f8a90178c62af62f40b" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 35523 } }
I20260807 08:47:40.059620 7123 leader_election.cc:288] T 40c8edc51b414cb0ac6aabe17cd5026f P 48b5111c255f4226b5ce348cc76a2d1b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 661ee1d16b2648d98bce6027b5132089 (127.4.159.130:45177), 2348d0b1f3294f8a90178c62af62f40b (127.4.159.131:35523)
I20260807 08:47:40.062186 7156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:40.059764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.131:35523 (local address 127.4.159.129:35825)
0807 08:47:40.059892 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0807 08:47:40.059901 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 08:47:40.059958 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:47:40.060082 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:47:40.060083 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:47:40.060085 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:47:40.060136 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:40.060138 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:40.060708 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:40.060709 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:40.061454 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:40.061456 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:40.061953 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:40.061954 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:40.062020 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:40.062023 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 08:47:40.062066 (+ 43us) client_negotiation.cc:240] Negotiation successful
0807 08:47:40.062102 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":61,"threads_started":1}
I20260807 08:47:40.062186 7112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:40.059830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.129:35825 (local address 127.4.159.131:35523)
0807 08:47:40.059874 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:40.059876 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:40.059963 (+ 87us) server_negotiation.cc:407] Connection header received
0807 08:47:40.060015 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:40.060016 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:40.060028 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:40.060073 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:47:40.060190 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:40.060643 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:40.061538 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:40.061887 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:40.061946 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:40.061954 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:40.062081 (+ 127us) server_negotiation.cc:299] Negotiation successful
0807 08:47:40.062116 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260807 08:47:40.062449 7030 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40c8edc51b414cb0ac6aabe17cd5026f" candidate_uuid: "48b5111c255f4226b5ce348cc76a2d1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2348d0b1f3294f8a90178c62af62f40b" is_pre_election: true
I20260807 08:47:40.062525 7153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:40.059764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.130:45177 (local address 127.4.159.129:48993)
0807 08:47:40.059950 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0807 08:47:40.059960 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 08:47:40.060027 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:47:40.060164 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:47:40.060165 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:47:40.060167 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:47:40.060239 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:40.060241 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:40.060869 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:40.060870 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:40.061585 (+ 715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:40.061587 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:40.062121 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:40.062122 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:40.062194 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:40.062197 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 08:47:40.062402 (+ 205us) client_negotiation.cc:240] Negotiation successful
0807 08:47:40.062440 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"mutex_wait_us":12,"thread_start_us":50,"threads_started":1}
I20260807 08:47:40.062528 7157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:40.059829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.129:48993 (local address 127.4.159.130:45177)
0807 08:47:40.059935 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:40.059937 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:40.060025 (+ 88us) server_negotiation.cc:407] Connection header received
0807 08:47:40.060081 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:40.060082 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:40.060093 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:40.060155 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:47:40.060300 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:40.060792 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:40.061679 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:40.062047 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:40.062115 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:40.062123 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:40.062422 (+ 299us) server_negotiation.cc:299] Negotiation successful
0807 08:47:40.062467 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88,"thread_start_us":61,"threads_started":1}
I20260807 08:47:40.062587 7030 raft_consensus.cc:2454] T 40c8edc51b414cb0ac6aabe17cd5026f P 2348d0b1f3294f8a90178c62af62f40b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48b5111c255f4226b5ce348cc76a2d1b in term 0.
I20260807 08:47:40.062737 6817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40c8edc51b414cb0ac6aabe17cd5026f" candidate_uuid: "48b5111c255f4226b5ce348cc76a2d1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "661ee1d16b2648d98bce6027b5132089" is_pre_election: true
I20260807 08:47:40.062772 6514 leader_election.cc:302] T 40c8edc51b414cb0ac6aabe17cd5026f P 48b5111c255f4226b5ce348cc76a2d1b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2348d0b1f3294f8a90178c62af62f40b, 48b5111c255f4226b5ce348cc76a2d1b; no voters:
I20260807 08:47:40.062832 6817 raft_consensus.cc:2454] T 40c8edc51b414cb0ac6aabe17cd5026f P 661ee1d16b2648d98bce6027b5132089 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48b5111c255f4226b5ce348cc76a2d1b in term 0.
I20260807 08:47:40.062884 7123 raft_consensus.cc:2781] T 40c8edc51b414cb0ac6aabe17cd5026f P 48b5111c255f4226b5ce348cc76a2d1b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:47:40.062927 7123 raft_consensus.cc:482] T 40c8edc51b414cb0ac6aabe17cd5026f P 48b5111c255f4226b5ce348cc76a2d1b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:47:40.062947 7123 raft_consensus.cc:3037] T 40c8edc51b414cb0ac6aabe17cd5026f P 48b5111c255f4226b5ce348cc76a2d1b [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:47:40.063576 7123 raft_consensus.cc:504] T 40c8edc51b414cb0ac6aabe17cd5026f P 48b5111c255f4226b5ce348cc76a2d1b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b5111c255f4226b5ce348cc76a2d1b" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 44539 } } peers { permanent_uuid: "661ee1d16b2648d98bce6027b5132089" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 45177 } } peers { permanent_uuid: "2348d0b1f3294f8a90178c62af62f40b" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 35523 } }
I20260807 08:47:40.063695 7123 leader_election.cc:288] T 40c8edc51b414cb0ac6aabe17cd5026f P 48b5111c255f4226b5ce348cc76a2d1b [CANDIDATE]: Term 1 election: Requested vote from peers 661ee1d16b2648d98bce6027b5132089 (127.4.159.130:45177), 2348d0b1f3294f8a90178c62af62f40b (127.4.159.131:35523)
I20260807 08:47:40.063824 6817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40c8edc51b414cb0ac6aabe17cd5026f" candidate_uuid: "48b5111c255f4226b5ce348cc76a2d1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "661ee1d16b2648d98bce6027b5132089"
I20260807 08:47:40.063897 6817 raft_consensus.cc:3037] T 40c8edc51b414cb0ac6aabe17cd5026f P 661ee1d16b2648d98bce6027b5132089 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:47:40.063871 7030 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40c8edc51b414cb0ac6aabe17cd5026f" candidate_uuid: "48b5111c255f4226b5ce348cc76a2d1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2348d0b1f3294f8a90178c62af62f40b"
I20260807 08:47:40.063933 7030 raft_consensus.cc:3037] T 40c8edc51b414cb0ac6aabe17cd5026f P 2348d0b1f3294f8a90178c62af62f40b [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:47:40.064463 6817 raft_consensus.cc:2454] T 40c8edc51b414cb0ac6aabe17cd5026f P 661ee1d16b2648d98bce6027b5132089 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48b5111c255f4226b5ce348cc76a2d1b in term 1.
I20260807 08:47:40.064553 7030 raft_consensus.cc:2454] T 40c8edc51b414cb0ac6aabe17cd5026f P 2348d0b1f3294f8a90178c62af62f40b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48b5111c255f4226b5ce348cc76a2d1b in term 1.
I20260807 08:47:40.064654 6515 leader_election.cc:302] T 40c8edc51b414cb0ac6aabe17cd5026f P 48b5111c255f4226b5ce348cc76a2d1b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48b5111c255f4226b5ce348cc76a2d1b, 661ee1d16b2648d98bce6027b5132089; no voters:
I20260807 08:47:40.064756 7123 raft_consensus.cc:2781] T 40c8edc51b414cb0ac6aabe17cd5026f P 48b5111c255f4226b5ce348cc76a2d1b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:47:40.064906 7123 raft_consensus.cc:686] T 40c8edc51b414cb0ac6aabe17cd5026f P 48b5111c255f4226b5ce348cc76a2d1b [term 1 LEADER]: Becoming Leader. State: Replica: 48b5111c255f4226b5ce348cc76a2d1b, State: Running, Role: LEADER
I20260807 08:47:40.064971 7123 consensus_queue.cc:237] T 40c8edc51b414cb0ac6aabe17cd5026f P 48b5111c255f4226b5ce348cc76a2d1b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b5111c255f4226b5ce348cc76a2d1b" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 44539 } } peers { permanent_uuid: "661ee1d16b2648d98bce6027b5132089" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 45177 } } peers { permanent_uuid: "2348d0b1f3294f8a90178c62af62f40b" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 35523 } }
I20260807 08:47:40.065609 6388 catalog_manager.cc:5495] T 40c8edc51b414cb0ac6aabe17cd5026f P 48b5111c255f4226b5ce348cc76a2d1b reported cstate change: term changed from 0 to 1, leader changed from <none> to 48b5111c255f4226b5ce348cc76a2d1b (127.4.159.129). New cstate: current_term: 1 leader_uuid: "48b5111c255f4226b5ce348cc76a2d1b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b5111c255f4226b5ce348cc76a2d1b" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 44539 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "661ee1d16b2648d98bce6027b5132089" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 45177 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2348d0b1f3294f8a90178c62af62f40b" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 35523 } health_report { overall_health: UNKNOWN } } }
Aug 07 08:47:40 dist-test-slave-6zbq krb5kdc[6315](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092458, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.190@KRBTEST.COM for kudu/127.4.159.190@KRBTEST.COM
I20260807 08:47:40.172794 7171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:40.166876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:43663 (local address 127.0.0.1:35654)
0807 08:47:40.167004 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0807 08:47:40.167015 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 08:47:40.167070 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:47:40.167190 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:47:40.167190 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:47:40.167466 (+ 276us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:47:40.167505 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:40.167507 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:40.168085 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:40.168086 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:40.168584 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:40.168585 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:40.168850 (+ 265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:40.168851 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:40.168884 (+ 33us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:40.169276 (+ 392us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:47:40.169280 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:47:40.170588 (+ 1308us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:47:40.171898 (+ 1310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:47:40.171905 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:47:40.171907 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:47:40.172101 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:47:40.172277 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:47:40.172278 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:47:40.172278 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:47:40.172338 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:47:40.172544 (+ 206us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:47:40.172550 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:47:40.172587 (+ 37us) client_negotiation.cc:769] Sending connection context
0807 08:47:40.172632 (+ 45us) client_negotiation.cc:240] Negotiation successful
0807 08:47:40.172719 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":61,"threads_started":1}
I20260807 08:47:40.172858 7172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:40.166936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35654 (local address 127.4.159.190:43663)
0807 08:47:40.167061 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:40.167063 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:40.167071 (+ 8us) server_negotiation.cc:407] Connection header received
0807 08:47:40.167120 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:40.167121 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:40.167130 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:40.167177 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:47:40.167557 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:40.168035 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:40.168634 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:40.168807 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:40.168846 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:40.170665 (+ 1819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:47:40.170667 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:47:40.170667 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:47:40.170673 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:47:40.171834 (+ 1161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:40.172156 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:40.172157 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:40.172157 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:40.172209 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:40.172396 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:40.172397 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:40.172397 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:40.172500 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:47:40.172550 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:40.172655 (+ 105us) server_negotiation.cc:299] Negotiation successful
0807 08:47:40.172766 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":44,"server-negotiator.queue_time_us":105,"thread_start_us":83,"threads_started":1}
I20260807 08:47:40.174952 6388 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.4.159.190@KRBTEST.COM'} at 127.0.0.1:35654:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 08:47:40.175200 6388 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:47:40.176527 6792 tablet_service.cc:1467] Processing CreateTablet for tablet 1e3729ab00604e1a805124c00f3cac41 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=78fca255f9f1462e91026d09e0704b7d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 08:47:40.176610 6996 tablet_service.cc:1467] Processing CreateTablet for tablet 1e3729ab00604e1a805124c00f3cac41 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=78fca255f9f1462e91026d09e0704b7d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 08:47:40.176627 6584 tablet_service.cc:1467] Processing CreateTablet for tablet 1e3729ab00604e1a805124c00f3cac41 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=78fca255f9f1462e91026d09e0704b7d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 08:47:40.176720 6996 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e3729ab00604e1a805124c00f3cac41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:47:40.176718 6792 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e3729ab00604e1a805124c00f3cac41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:47:40.176764 6584 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e3729ab00604e1a805124c00f3cac41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:47:40.178130 7174 tablet_bootstrap.cc:492] T 1e3729ab00604e1a805124c00f3cac41 P 48b5111c255f4226b5ce348cc76a2d1b: Bootstrap starting.
I20260807 08:47:40.178237 7180 tablet_bootstrap.cc:492] T 1e3729ab00604e1a805124c00f3cac41 P 2348d0b1f3294f8a90178c62af62f40b: Bootstrap starting.
I20260807 08:47:40.178324 7183 tablet_bootstrap.cc:492] T 1e3729ab00604e1a805124c00f3cac41 P 661ee1d16b2648d98bce6027b5132089: Bootstrap starting.
I20260807 08:47:40.178660 7180 tablet_bootstrap.cc:654] T 1e3729ab00604e1a805124c00f3cac41 P 2348d0b1f3294f8a90178c62af62f40b: Neither blocks nor log segments found. Creating new log.
I20260807 08:47:40.178668 7183 tablet_bootstrap.cc:654] T 1e3729ab00604e1a805124c00f3cac41 P 661ee1d16b2648d98bce6027b5132089: Neither blocks nor log segments found. Creating new log.
I20260807 08:47:40.178660 7174 tablet_bootstrap.cc:654] T 1e3729ab00604e1a805124c00f3cac41 P 48b5111c255f4226b5ce348cc76a2d1b: Neither blocks nor log segments found. Creating new log.
I20260807 08:47:40.179222 7174 tablet_bootstrap.cc:492] T 1e3729ab00604e1a805124c00f3cac41 P 48b5111c255f4226b5ce348cc76a2d1b: No bootstrap required, opened a new log
I20260807 08:47:40.179231 7180 tablet_bootstrap.cc:492] T 1e3729ab00604e1a805124c00f3cac41 P 2348d0b1f3294f8a90178c62af62f40b: No bootstrap required, opened a new log
I20260807 08:47:40.179262 7174 ts_tablet_manager.cc:1397] T 1e3729ab00604e1a805124c00f3cac41 P 48b5111c255f4226b5ce348cc76a2d1b: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 08:47:40.179275 7180 ts_tablet_manager.cc:1397] T 1e3729ab00604e1a805124c00f3cac41 P 2348d0b1f3294f8a90178c62af62f40b: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 08:47:40.179394 7174 raft_consensus.cc:348] T 1e3729ab00604e1a805124c00f3cac41 P 48b5111c255f4226b5ce348cc76a2d1b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b5111c255f4226b5ce348cc76a2d1b" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 44539 } } peers { permanent_uuid: "661ee1d16b2648d98bce6027b5132089" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 45177 } } peers { permanent_uuid: "2348d0b1f3294f8a90178c62af62f40b" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 35523 } }
I20260807 08:47:40.179451 7174 raft_consensus.cc:374] T 1e3729ab00604e1a805124c00f3cac41 P 48b5111c255f4226b5ce348cc76a2d1b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:47:40.179415 7180 raft_consensus.cc:348] T 1e3729ab00604e1a805124c00f3cac41 P 2348d0b1f3294f8a90178c62af62f40b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b5111c255f4226b5ce348cc76a2d1b" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 44539 } } peers { permanent_uuid: "661ee1d16b2648d98bce6027b5132089" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 45177 } } peers { permanent_uuid: "2348d0b1f3294f8a90178c62af62f40b" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 35523 } }
I20260807 08:47:40.179464 7174 raft_consensus.cc:729] T 1e3729ab00604e1a805124c00f3cac41 P 48b5111c255f4226b5ce348cc76a2d1b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48b5111c255f4226b5ce348cc76a2d1b, State: Initialized, Role: FOLLOWER
I20260807 08:47:40.179468 7180 raft_consensus.cc:374] T 1e3729ab00604e1a805124c00f3cac41 P 2348d0b1f3294f8a90178c62af62f40b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:47:40.179487 7180 raft_consensus.cc:729] T 1e3729ab00604e1a805124c00f3cac41 P 2348d0b1f3294f8a90178c62af62f40b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2348d0b1f3294f8a90178c62af62f40b, State: Initialized, Role: FOLLOWER
I20260807 08:47:40.179497 7174 consensus_queue.cc:260] T 1e3729ab00604e1a805124c00f3cac41 P 48b5111c255f4226b5ce348cc76a2d1b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b5111c255f4226b5ce348cc76a2d1b" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 44539 } } peers { permanent_uuid: "661ee1d16b2648d98bce6027b5132089" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 45177 } } peers { permanent_uuid: "2348d0b1f3294f8a90178c62af62f40b" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 35523 } }
I20260807 08:47:40.179536 7180 consensus_queue.cc:260] T 1e3729ab00604e1a805124c00f3cac41 P 2348d0b1f3294f8a90178c62af62f40b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b5111c255f4226b5ce348cc76a2d1b" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 44539 } } peers { permanent_uuid: "661ee1d16b2648d98bce6027b5132089" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 45177 } } peers { permanent_uuid: "2348d0b1f3294f8a90178c62af62f40b" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 35523 } }
I20260807 08:47:40.179678 7180 ts_tablet_manager.cc:1428] T 1e3729ab00604e1a805124c00f3cac41 P 2348d0b1f3294f8a90178c62af62f40b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:40.179643 7123 tablet_replica.cc:439] T 1e3729ab00604e1a805124c00f3cac41 P 48b5111c255f4226b5ce348cc76a2d1b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b5111c255f4226b5ce348cc76a2d1b" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 44539 } } peers { permanent_uuid: "661ee1d16b2648d98bce6027b5132089" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 45177 } } peers { permanent_uuid: "2348d0b1f3294f8a90178c62af62f40b" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 35523 } } }
I20260807 08:47:40.179719 7123 tablet_replica.cc:442] T 1e3729ab00604e1a805124c00f3cac41 P 48b5111c255f4226b5ce348cc76a2d1b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 08:47:40.179606 7174 ts_tablet_manager.cc:1428] T 1e3729ab00604e1a805124c00f3cac41 P 48b5111c255f4226b5ce348cc76a2d1b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:40.179885 7183 tablet_bootstrap.cc:492] T 1e3729ab00604e1a805124c00f3cac41 P 661ee1d16b2648d98bce6027b5132089: No bootstrap required, opened a new log
I20260807 08:47:40.179931 7183 ts_tablet_manager.cc:1397] T 1e3729ab00604e1a805124c00f3cac41 P 661ee1d16b2648d98bce6027b5132089: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260807 08:47:40.179872 7189 tablet_replica.cc:439] T 1e3729ab00604e1a805124c00f3cac41 P 2348d0b1f3294f8a90178c62af62f40b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b5111c255f4226b5ce348cc76a2d1b" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 44539 } } peers { permanent_uuid: "661ee1d16b2648d98bce6027b5132089" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 45177 } } peers { permanent_uuid: "2348d0b1f3294f8a90178c62af62f40b" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 35523 } } }
I20260807 08:47:40.180019 7189 tablet_replica.cc:442] T 1e3729ab00604e1a805124c00f3cac41 P 2348d0b1f3294f8a90178c62af62f40b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 08:47:40.180075 7183 raft_consensus.cc:348] T 1e3729ab00604e1a805124c00f3cac41 P 661ee1d16b2648d98bce6027b5132089 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b5111c255f4226b5ce348cc76a2d1b" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 44539 } } peers { permanent_uuid: "661ee1d16b2648d98bce6027b5132089" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 45177 } } peers { permanent_uuid: "2348d0b1f3294f8a90178c62af62f40b" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 35523 } }
I20260807 08:47:40.180142 7183 raft_consensus.cc:374] T 1e3729ab00604e1a805124c00f3cac41 P 661ee1d16b2648d98bce6027b5132089 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:47:40.180168 7183 raft_consensus.cc:729] T 1e3729ab00604e1a805124c00f3cac41 P 661ee1d16b2648d98bce6027b5132089 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 661ee1d16b2648d98bce6027b5132089, State: Initialized, Role: FOLLOWER
I20260807 08:47:40.180212 7183 consensus_queue.cc:260] T 1e3729ab00604e1a805124c00f3cac41 P 661ee1d16b2648d98bce6027b5132089 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b5111c255f4226b5ce348cc76a2d1b" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 44539 } } peers { permanent_uuid: "661ee1d16b2648d98bce6027b5132089" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 45177 } } peers { permanent_uuid: "2348d0b1f3294f8a90178c62af62f40b" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 35523 } }
I20260807 08:47:40.180359 7183 ts_tablet_manager.cc:1428] T 1e3729ab00604e1a805124c00f3cac41 P 661ee1d16b2648d98bce6027b5132089: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260807 08:47:40.180444 7190 tablet_replica.cc:439] T 1e3729ab00604e1a805124c00f3cac41 P 661ee1d16b2648d98bce6027b5132089: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b5111c255f4226b5ce348cc76a2d1b" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 44539 } } peers { permanent_uuid: "661ee1d16b2648d98bce6027b5132089" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 45177 } } peers { permanent_uuid: "2348d0b1f3294f8a90178c62af62f40b" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 35523 } } }
I20260807 08:47:40.180596 7190 tablet_replica.cc:442] T 1e3729ab00604e1a805124c00f3cac41 P 661ee1d16b2648d98bce6027b5132089: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 08:47:40.221933 7189 raft_consensus.cc:482] T 1e3729ab00604e1a805124c00f3cac41 P 2348d0b1f3294f8a90178c62af62f40b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:47:40.221988 7189 raft_consensus.cc:504] T 1e3729ab00604e1a805124c00f3cac41 P 2348d0b1f3294f8a90178c62af62f40b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b5111c255f4226b5ce348cc76a2d1b" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 44539 } } peers { permanent_uuid: "661ee1d16b2648d98bce6027b5132089" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 45177 } } peers { permanent_uuid: "2348d0b1f3294f8a90178c62af62f40b" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 35523 } }
I20260807 08:47:40.222122 7189 leader_election.cc:288] T 1e3729ab00604e1a805124c00f3cac41 P 2348d0b1f3294f8a90178c62af62f40b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48b5111c255f4226b5ce348cc76a2d1b (127.4.159.129:44539), 661ee1d16b2648d98bce6027b5132089 (127.4.159.130:45177)
I20260807 08:47:40.222273 6817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e3729ab00604e1a805124c00f3cac41" candidate_uuid: "2348d0b1f3294f8a90178c62af62f40b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "661ee1d16b2648d98bce6027b5132089" is_pre_election: true
I20260807 08:47:40.222280 6614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e3729ab00604e1a805124c00f3cac41" candidate_uuid: "2348d0b1f3294f8a90178c62af62f40b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48b5111c255f4226b5ce348cc76a2d1b" is_pre_election: true
I20260807 08:47:40.222362 6614 raft_consensus.cc:2454] T 1e3729ab00604e1a805124c00f3cac41 P 48b5111c255f4226b5ce348cc76a2d1b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2348d0b1f3294f8a90178c62af62f40b in term 0.
I20260807 08:47:40.222364 6817 raft_consensus.cc:2454] T 1e3729ab00604e1a805124c00f3cac41 P 661ee1d16b2648d98bce6027b5132089 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2348d0b1f3294f8a90178c62af62f40b in term 0.
I20260807 08:47:40.222503 6929 leader_election.cc:302] T 1e3729ab00604e1a805124c00f3cac41 P 2348d0b1f3294f8a90178c62af62f40b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2348d0b1f3294f8a90178c62af62f40b, 661ee1d16b2648d98bce6027b5132089; no voters:
I20260807 08:47:40.222604 7189 raft_consensus.cc:2781] T 1e3729ab00604e1a805124c00f3cac41 P 2348d0b1f3294f8a90178c62af62f40b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:47:40.222653 7189 raft_consensus.cc:482] T 1e3729ab00604e1a805124c00f3cac41 P 2348d0b1f3294f8a90178c62af62f40b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:47:40.222679 7189 raft_consensus.cc:3037] T 1e3729ab00604e1a805124c00f3cac41 P 2348d0b1f3294f8a90178c62af62f40b [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:47:40.223256 7189 raft_consensus.cc:504] T 1e3729ab00604e1a805124c00f3cac41 P 2348d0b1f3294f8a90178c62af62f40b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b5111c255f4226b5ce348cc76a2d1b" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 44539 } } peers { permanent_uuid: "661ee1d16b2648d98bce6027b5132089" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 45177 } } peers { permanent_uuid: "2348d0b1f3294f8a90178c62af62f40b" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 35523 } }
I20260807 08:47:40.223368 7189 leader_election.cc:288] T 1e3729ab00604e1a805124c00f3cac41 P 2348d0b1f3294f8a90178c62af62f40b [CANDIDATE]: Term 1 election: Requested vote from peers 48b5111c255f4226b5ce348cc76a2d1b (127.4.159.129:44539), 661ee1d16b2648d98bce6027b5132089 (127.4.159.130:45177)
I20260807 08:47:40.223496 6614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e3729ab00604e1a805124c00f3cac41" candidate_uuid: "2348d0b1f3294f8a90178c62af62f40b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48b5111c255f4226b5ce348cc76a2d1b"
I20260807 08:47:40.223518 6817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e3729ab00604e1a805124c00f3cac41" candidate_uuid: "2348d0b1f3294f8a90178c62af62f40b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "661ee1d16b2648d98bce6027b5132089"
I20260807 08:47:40.223563 6614 raft_consensus.cc:3037] T 1e3729ab00604e1a805124c00f3cac41 P 48b5111c255f4226b5ce348cc76a2d1b [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:47:40.223568 6817 raft_consensus.cc:3037] T 1e3729ab00604e1a805124c00f3cac41 P 661ee1d16b2648d98bce6027b5132089 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:47:40.224190 6614 raft_consensus.cc:2454] T 1e3729ab00604e1a805124c00f3cac41 P 48b5111c255f4226b5ce348cc76a2d1b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2348d0b1f3294f8a90178c62af62f40b in term 1.
I20260807 08:47:40.224190 6817 raft_consensus.cc:2454] T 1e3729ab00604e1a805124c00f3cac41 P 661ee1d16b2648d98bce6027b5132089 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2348d0b1f3294f8a90178c62af62f40b in term 1.
I20260807 08:47:40.224352 6928 leader_election.cc:302] T 1e3729ab00604e1a805124c00f3cac41 P 2348d0b1f3294f8a90178c62af62f40b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2348d0b1f3294f8a90178c62af62f40b, 48b5111c255f4226b5ce348cc76a2d1b; no voters:
I20260807 08:47:40.224453 7189 raft_consensus.cc:2781] T 1e3729ab00604e1a805124c00f3cac41 P 2348d0b1f3294f8a90178c62af62f40b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:47:40.224556 7189 raft_consensus.cc:686] T 1e3729ab00604e1a805124c00f3cac41 P 2348d0b1f3294f8a90178c62af62f40b [term 1 LEADER]: Becoming Leader. State: Replica: 2348d0b1f3294f8a90178c62af62f40b, State: Running, Role: LEADER
I20260807 08:47:40.224607 7189 consensus_queue.cc:237] T 1e3729ab00604e1a805124c00f3cac41 P 2348d0b1f3294f8a90178c62af62f40b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b5111c255f4226b5ce348cc76a2d1b" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 44539 } } peers { permanent_uuid: "661ee1d16b2648d98bce6027b5132089" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 45177 } } peers { permanent_uuid: "2348d0b1f3294f8a90178c62af62f40b" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 35523 } }
I20260807 08:47:40.224855 7191 tablet_replica.cc:439] T 1e3729ab00604e1a805124c00f3cac41 P 2348d0b1f3294f8a90178c62af62f40b: TxnStatusTablet state changed. Reason: New leader 2348d0b1f3294f8a90178c62af62f40b. Latest consensus state: current_term: 1 leader_uuid: "2348d0b1f3294f8a90178c62af62f40b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b5111c255f4226b5ce348cc76a2d1b" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 44539 } } peers { permanent_uuid: "661ee1d16b2648d98bce6027b5132089" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 45177 } } peers { permanent_uuid: "2348d0b1f3294f8a90178c62af62f40b" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 35523 } } }
I20260807 08:47:40.224936 7191 tablet_replica.cc:442] T 1e3729ab00604e1a805124c00f3cac41 P 2348d0b1f3294f8a90178c62af62f40b: This TxnStatusTablet replica's current role is: LEADER
I20260807 08:47:40.225100 6388 catalog_manager.cc:5495] T 1e3729ab00604e1a805124c00f3cac41 P 2348d0b1f3294f8a90178c62af62f40b reported cstate change: term changed from 0 to 1, leader changed from <none> to 2348d0b1f3294f8a90178c62af62f40b (127.4.159.131). New cstate: current_term: 1 leader_uuid: "2348d0b1f3294f8a90178c62af62f40b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b5111c255f4226b5ce348cc76a2d1b" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 44539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "661ee1d16b2648d98bce6027b5132089" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 45177 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2348d0b1f3294f8a90178c62af62f40b" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 35523 } health_report { overall_health: HEALTHY } } }
I20260807 08:47:40.225337 7193 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 08:47:40.225415 7193 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 08:47:40.237320 7112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:40.235005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55830 (local address 127.4.159.131:35523)
0807 08:47:40.235043 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:40.235045 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:40.235084 (+ 39us) server_negotiation.cc:407] Connection header received
0807 08:47:40.235171 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:40.235172 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:40.235178 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:40.235221 (+ 43us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:47:40.235327 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:40.235753 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:40.236773 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:40.236941 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:40.236982 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:40.237147 (+ 165us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:47:40.237169 (+ 22us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:47:40.237199 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:40.237257 (+ 58us) server_negotiation.cc:299] Negotiation successful
0807 08:47:40.237271 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260807 08:47:40.237365 7171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:40.234961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.131:35523 (local address 127.0.0.1:55830)
0807 08:47:40.235041 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0807 08:47:40.235051 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 08:47:40.235105 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:47:40.235229 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:47:40.235230 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:47:40.235231 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 08:47:40.235273 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:40.235275 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:40.235809 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:40.235810 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:40.236705 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:40.236708 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:40.236995 (+ 287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:40.236996 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:40.237057 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:40.237073 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 08:47:40.237202 (+ 129us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 08:47:40.237207 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 08:47:40.237253 (+ 46us) client_negotiation.cc:240] Negotiation successful
0807 08:47:40.237295 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260807 08:47:40.238279 6817 raft_consensus.cc:1261] T 1e3729ab00604e1a805124c00f3cac41 P 661ee1d16b2648d98bce6027b5132089 [term 1 FOLLOWER]: Refusing update from remote peer 2348d0b1f3294f8a90178c62af62f40b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:47:40.238287 6614 raft_consensus.cc:1261] T 1e3729ab00604e1a805124c00f3cac41 P 48b5111c255f4226b5ce348cc76a2d1b [term 1 FOLLOWER]: Refusing update from remote peer 2348d0b1f3294f8a90178c62af62f40b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:47:40.238499 7191 consensus_queue.cc:1035] T 1e3729ab00604e1a805124c00f3cac41 P 2348d0b1f3294f8a90178c62af62f40b [LEADER]: Connected to new peer: Peer: permanent_uuid: "661ee1d16b2648d98bce6027b5132089" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 45177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:47:40.238584 7189 consensus_queue.cc:1035] T 1e3729ab00604e1a805124c00f3cac41 P 2348d0b1f3294f8a90178c62af62f40b [LEADER]: Connected to new peer: Peer: permanent_uuid: "48b5111c255f4226b5ce348cc76a2d1b" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 44539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:47:40.239023 7123 tablet_replica.cc:439] T 1e3729ab00604e1a805124c00f3cac41 P 48b5111c255f4226b5ce348cc76a2d1b: TxnStatusTablet state changed. Reason: New leader 2348d0b1f3294f8a90178c62af62f40b. Latest consensus state: current_term: 1 leader_uuid: "2348d0b1f3294f8a90178c62af62f40b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b5111c255f4226b5ce348cc76a2d1b" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 44539 } } peers { permanent_uuid: "661ee1d16b2648d98bce6027b5132089" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 45177 } } peers { permanent_uuid: "2348d0b1f3294f8a90178c62af62f40b" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 35523 } } }
I20260807 08:47:40.239101 7123 tablet_replica.cc:442] T 1e3729ab00604e1a805124c00f3cac41 P 48b5111c255f4226b5ce348cc76a2d1b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 08:47:40.239046 7190 tablet_replica.cc:439] T 1e3729ab00604e1a805124c00f3cac41 P 661ee1d16b2648d98bce6027b5132089: TxnStatusTablet state changed. Reason: New leader 2348d0b1f3294f8a90178c62af62f40b. Latest consensus state: current_term: 1 leader_uuid: "2348d0b1f3294f8a90178c62af62f40b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b5111c255f4226b5ce348cc76a2d1b" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 44539 } } peers { permanent_uuid: "661ee1d16b2648d98bce6027b5132089" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 45177 } } peers { permanent_uuid: "2348d0b1f3294f8a90178c62af62f40b" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 35523 } } }
I20260807 08:47:40.239149 7190 tablet_replica.cc:442] T 1e3729ab00604e1a805124c00f3cac41 P 661ee1d16b2648d98bce6027b5132089: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 08:47:40.239840 7191 tablet_replica.cc:439] T 1e3729ab00604e1a805124c00f3cac41 P 2348d0b1f3294f8a90178c62af62f40b: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2348d0b1f3294f8a90178c62af62f40b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b5111c255f4226b5ce348cc76a2d1b" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 44539 } } peers { permanent_uuid: "661ee1d16b2648d98bce6027b5132089" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 45177 } } peers { permanent_uuid: "2348d0b1f3294f8a90178c62af62f40b" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 35523 } } }
I20260807 08:47:40.239914 7191 tablet_replica.cc:442] T 1e3729ab00604e1a805124c00f3cac41 P 2348d0b1f3294f8a90178c62af62f40b: This TxnStatusTablet replica's current role is: LEADER
I20260807 08:47:40.239948 7189 tablet_replica.cc:439] T 1e3729ab00604e1a805124c00f3cac41 P 2348d0b1f3294f8a90178c62af62f40b: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2348d0b1f3294f8a90178c62af62f40b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b5111c255f4226b5ce348cc76a2d1b" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 44539 } } peers { permanent_uuid: "661ee1d16b2648d98bce6027b5132089" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 45177 } } peers { permanent_uuid: "2348d0b1f3294f8a90178c62af62f40b" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 35523 } } }
I20260807 08:47:40.240029 7189 tablet_replica.cc:442] T 1e3729ab00604e1a805124c00f3cac41 P 2348d0b1f3294f8a90178c62af62f40b: This TxnStatusTablet replica's current role is: LEADER
I20260807 08:47:40.239961 7198 mvcc.cc:205] Tried to move back new op lower bound from 7315834717134733312 to 7315834717080252416. Current Snapshot: MvccSnapshot[applied={T|T < 7315834717134733312}]
I20260807 08:47:40.240155 7190 tablet_replica.cc:439] T 1e3729ab00604e1a805124c00f3cac41 P 661ee1d16b2648d98bce6027b5132089: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2348d0b1f3294f8a90178c62af62f40b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b5111c255f4226b5ce348cc76a2d1b" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 44539 } } peers { permanent_uuid: "661ee1d16b2648d98bce6027b5132089" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 45177 } } peers { permanent_uuid: "2348d0b1f3294f8a90178c62af62f40b" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 35523 } } }
I20260807 08:47:40.240228 7190 tablet_replica.cc:442] T 1e3729ab00604e1a805124c00f3cac41 P 661ee1d16b2648d98bce6027b5132089: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 08:47:40.240319 7201 mvcc.cc:205] Tried to move back new op lower bound from 7315834717134733312 to 7315834717080252416. Current Snapshot: MvccSnapshot[applied={T|T < 7315834717134733312}]
I20260807 08:47:40.240556 7200 mvcc.cc:205] Tried to move back new op lower bound from 7315834717134733312 to 7315834717080252416. Current Snapshot: MvccSnapshot[applied={T|T < 7315834717134733312}]
I20260807 08:47:40.240605 7123 tablet_replica.cc:439] T 1e3729ab00604e1a805124c00f3cac41 P 48b5111c255f4226b5ce348cc76a2d1b: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2348d0b1f3294f8a90178c62af62f40b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b5111c255f4226b5ce348cc76a2d1b" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 44539 } } peers { permanent_uuid: "661ee1d16b2648d98bce6027b5132089" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 45177 } } peers { permanent_uuid: "2348d0b1f3294f8a90178c62af62f40b" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 35523 } } }
I20260807 08:47:40.240662 7123 tablet_replica.cc:442] T 1e3729ab00604e1a805124c00f3cac41 P 48b5111c255f4226b5ce348cc76a2d1b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 08:47:40.244954 7211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:40.242698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44740 (local address 127.4.159.129:44539)
0807 08:47:40.242833 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:40.242835 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:40.242870 (+ 35us) server_negotiation.cc:407] Connection header received
0807 08:47:40.242947 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:40.242947 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:40.242965 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:40.243031 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:47:40.243130 (+ 99us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:40.243588 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:40.244260 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:40.244458 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:40.244508 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:40.244650 (+ 142us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:47:40.244742 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:47:40.244805 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:40.244841 (+ 36us) server_negotiation.cc:299] Negotiation successful
0807 08:47:40.244861 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":68,"threads_started":1}
I20260807 08:47:40.249300 7112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:40.247136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.129:39581 (local address 127.4.159.131:35523)
0807 08:47:40.247173 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:40.247174 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:40.247294 (+ 120us) server_negotiation.cc:407] Connection header received
0807 08:47:40.247352 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:40.247353 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:40.247363 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:40.247404 (+ 41us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:47:40.247510 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:40.248012 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:40.248693 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:40.248891 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:40.248940 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:40.249098 (+ 158us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:47:40.249128 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:47:40.249169 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:40.249222 (+ 53us) server_negotiation.cc:299] Negotiation successful
0807 08:47:40.249240 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260807 08:47:40.249346 7215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:40.247091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.131:35523 (local address 127.4.159.129:39581)
0807 08:47:40.247233 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0807 08:47:40.247242 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 08:47:40.247304 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:47:40.247405 (+ 101us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:47:40.247406 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:47:40.247407 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 08:47:40.247460 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:40.247461 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:40.248079 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:40.248080 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:40.248626 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:40.248627 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:40.248953 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:40.248954 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:40.249024 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:40.249037 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 08:47:40.249181 (+ 144us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 08:47:40.249182 (+ 1us) client_negotiation.cc:769] Sending connection context
0807 08:47:40.249222 (+ 40us) client_negotiation.cc:240] Negotiation successful
0807 08:47:40.249259 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":73,"threads_started":1}
I20260807 08:47:40.251426 6817 raft_consensus.cc:1261] T 40c8edc51b414cb0ac6aabe17cd5026f P 661ee1d16b2648d98bce6027b5132089 [term 1 FOLLOWER]: Refusing update from remote peer 48b5111c255f4226b5ce348cc76a2d1b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:47:40.251464 7030 raft_consensus.cc:1261] T 40c8edc51b414cb0ac6aabe17cd5026f P 2348d0b1f3294f8a90178c62af62f40b [term 1 FOLLOWER]: Refusing update from remote peer 48b5111c255f4226b5ce348cc76a2d1b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:47:40.251655 7160 consensus_queue.cc:1035] T 40c8edc51b414cb0ac6aabe17cd5026f P 48b5111c255f4226b5ce348cc76a2d1b [LEADER]: Connected to new peer: Peer: permanent_uuid: "661ee1d16b2648d98bce6027b5132089" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 45177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:47:40.251771 7123 consensus_queue.cc:1035] T 40c8edc51b414cb0ac6aabe17cd5026f P 48b5111c255f4226b5ce348cc76a2d1b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2348d0b1f3294f8a90178c62af62f40b" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 35523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:47:40.260178 7211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:40.257691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.131:35153 (local address 127.4.159.129:44539)
0807 08:47:40.257736 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:40.257737 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:40.257784 (+ 47us) server_negotiation.cc:407] Connection header received
0807 08:47:40.257838 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:40.257839 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:40.257846 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:40.257884 (+ 38us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:47:40.257974 (+ 90us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:40.258423 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:40.259474 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:40.259764 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:40.259801 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:40.259995 (+ 194us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:47:40.260019 (+ 24us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:47:40.260056 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:40.260122 (+ 66us) server_negotiation.cc:299] Negotiation successful
0807 08:47:40.260137 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260807 08:47:40.260233 7225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:40.257528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.129:44539 (local address 127.4.159.131:35153)
0807 08:47:40.257726 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0807 08:47:40.257734 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0807 08:47:40.257791 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:47:40.257893 (+ 102us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:47:40.257894 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:47:40.257895 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 08:47:40.257933 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:40.257934 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:40.258484 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:40.258485 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:40.259374 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:40.259377 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:40.259854 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:40.259855 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:40.259908 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:40.259921 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 08:47:40.260063 (+ 142us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 08:47:40.260064 (+ 1us) client_negotiation.cc:769] Sending connection context
0807 08:47:40.260117 (+ 53us) client_negotiation.cc:240] Negotiation successful
0807 08:47:40.260156 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":68,"threads_started":1}
I20260807 08:47:40.270196 6384 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35644:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:47:40.270282 6384 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35644:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:47:40.270839 6384 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a9b26c4a4f35408cac8649134220bab5: Sending DeleteTablet for 3 replicas of tablet 40c8edc51b414cb0ac6aabe17cd5026f
I20260807 08:47:40.271051 6792 tablet_service.cc:1514] Processing DeleteTablet for tablet 40c8edc51b414cb0ac6aabe17cd5026f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:47:40 UTC) from {username='kudu', principal='kudu/127.4.159.190@KRBTEST.COM'} at 127.0.0.1:46088
I20260807 08:47:40.271140 6584 tablet_service.cc:1514] Processing DeleteTablet for tablet 40c8edc51b414cb0ac6aabe17cd5026f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:47:40 UTC) from {username='kudu', principal='kudu/127.4.159.190@KRBTEST.COM'} at 127.0.0.1:44724
I20260807 08:47:40.271333 6996 tablet_service.cc:1514] Processing DeleteTablet for tablet 40c8edc51b414cb0ac6aabe17cd5026f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:47:40 UTC) from {username='kudu', principal='kudu/127.4.159.190@KRBTEST.COM'} at 127.0.0.1:55818
I20260807 08:47:40.271998 7231 tablet_replica.cc:330] T 40c8edc51b414cb0ac6aabe17cd5026f P 661ee1d16b2648d98bce6027b5132089: stopping tablet replica
I20260807 08:47:40.272089 7231 raft_consensus.cc:2229] T 40c8edc51b414cb0ac6aabe17cd5026f P 661ee1d16b2648d98bce6027b5132089 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:47:40.272164 7231 raft_consensus.cc:2258] T 40c8edc51b414cb0ac6aabe17cd5026f P 661ee1d16b2648d98bce6027b5132089 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:47:40.272429 7231 ts_tablet_manager.cc:1905] T 40c8edc51b414cb0ac6aabe17cd5026f P 661ee1d16b2648d98bce6027b5132089: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:47:40.274073 7231 ts_tablet_manager.cc:1918] T 40c8edc51b414cb0ac6aabe17cd5026f P 661ee1d16b2648d98bce6027b5132089: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 08:47:40.274133 7231 log.cc:1196] T 40c8edc51b414cb0ac6aabe17cd5026f P 661ee1d16b2648d98bce6027b5132089: Deleting WAL directory at /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786092453486433-4734-0/minicluster-data/ts-1/wal/wals/40c8edc51b414cb0ac6aabe17cd5026f
I20260807 08:47:40.274366 7231 ts_tablet_manager.cc:1939] T 40c8edc51b414cb0ac6aabe17cd5026f P 661ee1d16b2648d98bce6027b5132089: Deleting consensus metadata
I20260807 08:47:40.274674 6364 catalog_manager.cc:4841] TS 661ee1d16b2648d98bce6027b5132089 (127.4.159.130:45177): tablet 40c8edc51b414cb0ac6aabe17cd5026f (table test-table [id=54a2778ed6f4448892dfb2dd31d87536]) successfully deleted
I20260807 08:47:40.278430 7232 tablet_replica.cc:330] T 40c8edc51b414cb0ac6aabe17cd5026f P 48b5111c255f4226b5ce348cc76a2d1b: stopping tablet replica
I20260807 08:47:40.278432 7233 tablet_replica.cc:330] T 40c8edc51b414cb0ac6aabe17cd5026f P 2348d0b1f3294f8a90178c62af62f40b: stopping tablet replica
I20260807 08:47:40.278494 7233 raft_consensus.cc:2229] T 40c8edc51b414cb0ac6aabe17cd5026f P 2348d0b1f3294f8a90178c62af62f40b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:47:40.278494 7232 raft_consensus.cc:2229] T 40c8edc51b414cb0ac6aabe17cd5026f P 48b5111c255f4226b5ce348cc76a2d1b [term 1 LEADER]: Raft consensus shutting down.
I20260807 08:47:40.278535 7233 raft_consensus.cc:2258] T 40c8edc51b414cb0ac6aabe17cd5026f P 2348d0b1f3294f8a90178c62af62f40b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:47:40.278589 7232 raft_consensus.cc:2258] T 40c8edc51b414cb0ac6aabe17cd5026f P 48b5111c255f4226b5ce348cc76a2d1b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:47:40.278759 7233 ts_tablet_manager.cc:1905] T 40c8edc51b414cb0ac6aabe17cd5026f P 2348d0b1f3294f8a90178c62af62f40b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:47:40.278919 7232 ts_tablet_manager.cc:1905] T 40c8edc51b414cb0ac6aabe17cd5026f P 48b5111c255f4226b5ce348cc76a2d1b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:47:40.280256 7233 ts_tablet_manager.cc:1918] T 40c8edc51b414cb0ac6aabe17cd5026f P 2348d0b1f3294f8a90178c62af62f40b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 08:47:40.280315 7233 log.cc:1196] T 40c8edc51b414cb0ac6aabe17cd5026f P 2348d0b1f3294f8a90178c62af62f40b: Deleting WAL directory at /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786092453486433-4734-0/minicluster-data/ts-2/wal/wals/40c8edc51b414cb0ac6aabe17cd5026f
I20260807 08:47:40.280485 7232 ts_tablet_manager.cc:1918] T 40c8edc51b414cb0ac6aabe17cd5026f P 48b5111c255f4226b5ce348cc76a2d1b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 08:47:40.280516 7233 ts_tablet_manager.cc:1939] T 40c8edc51b414cb0ac6aabe17cd5026f P 2348d0b1f3294f8a90178c62af62f40b: Deleting consensus metadata
I20260807 08:47:40.280580 7232 log.cc:1196] T 40c8edc51b414cb0ac6aabe17cd5026f P 48b5111c255f4226b5ce348cc76a2d1b: Deleting WAL directory at /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786092453486433-4734-0/minicluster-data/ts-0/wal/wals/40c8edc51b414cb0ac6aabe17cd5026f
I20260807 08:47:40.280792 6362 catalog_manager.cc:4841] TS 2348d0b1f3294f8a90178c62af62f40b (127.4.159.131:35523): tablet 40c8edc51b414cb0ac6aabe17cd5026f (table test-table [id=54a2778ed6f4448892dfb2dd31d87536]) successfully deleted
I20260807 08:47:40.280884 7232 ts_tablet_manager.cc:1939] T 40c8edc51b414cb0ac6aabe17cd5026f P 48b5111c255f4226b5ce348cc76a2d1b: Deleting consensus metadata
I20260807 08:47:40.281196 6362 catalog_manager.cc:4841] TS 48b5111c255f4226b5ce348cc76a2d1b (127.4.159.129:44539): tablet 40c8edc51b414cb0ac6aabe17cd5026f (table test-table [id=54a2778ed6f4448892dfb2dd31d87536]) successfully deleted
Aug 07 08:47:40 dist-test-slave-6zbq krb5kdc[6315](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092459, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.4.159.129@KRBTEST.COM
I20260807 08:47:40.290283 7211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:40.272064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44754 (local address 127.4.159.129:44539)
0807 08:47:40.272109 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:40.272110 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:40.280974 (+ 8864us) server_negotiation.cc:407] Connection header received
0807 08:47:40.281066 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:40.281067 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:40.281091 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:40.281143 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:47:40.281807 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:40.282542 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:40.283519 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:40.283821 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:40.283884 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:40.287147 (+ 3263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:47:40.287155 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:47:40.287156 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:47:40.287162 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:47:40.288944 (+ 1782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:40.289336 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:40.289339 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:40.289340 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:40.289425 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:40.289723 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:40.289724 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:40.289724 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:40.289901 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:47:40.290065 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:40.290120 (+ 55us) server_negotiation.cc:299] Negotiation successful
0807 08:47:40.290202 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
W20260807 08:47:40.290668 6672 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44754
I20260807 08:47:40.298846 7172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:40.291604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35662 (local address 127.4.159.190:43663)
0807 08:47:40.291652 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:40.291654 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:40.291662 (+ 8us) server_negotiation.cc:407] Connection header received
0807 08:47:40.291707 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:40.291708 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:40.291718 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:40.291745 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:47:40.292417 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:40.293155 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:40.294126 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:40.294443 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:40.294486 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:40.295809 (+ 1323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:47:40.295810 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:47:40.295811 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:47:40.295817 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:47:40.297365 (+ 1548us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:40.297791 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:40.297792 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:40.297793 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:40.297851 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:40.298129 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:40.298130 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:40.298130 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:40.298301 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:47:40.298452 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:40.298676 (+ 224us) server_negotiation.cc:299] Negotiation successful
0807 08:47:40.298760 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
W20260807 08:47:40.299366 6384 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35662
I20260807 08:47:40.299969 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 6487
I20260807 08:47:40.312487 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 6700
I20260807 08:47:40.321159 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 6903
I20260807 08:47:40.328789 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 6331
2026-08-07T08:47:40Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2899 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786092453486433-4734-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:47:40 dist-test-slave-6zbq krb5kdc[7240](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:47:40 dist-test-slave-6zbq krb5kdc[7240](info): set up 2 sockets
Aug 07 08:47:40 dist-test-slave-6zbq krb5kdc[7240](info): commencing operation
krb5kdc: starting...
W20260807 08:47:41.448578 4734 mini_kdc.cc:121] Time spent starting KDC: real 1.096s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786092453486433-4734-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786092453486433-4734-0/krb5kdc/test-user.keytab.
Aug 07 08:47:41 dist-test-slave-6zbq krb5kdc[7240](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092461, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:47:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:47:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.159.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.159.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.159.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab.
Entry for principal kudu/127.4.159.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab.
I20260807 08:47:41.553727 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786092453486433-4734-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786092453486433-4734-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786092453486433-4734-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.159.190:33213
--webserver_interface=127.4.159.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab
--principal=kudu/127.4.159.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:46163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.159.190:33213
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:47:41.631487 7261 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:47:41.631661 7261 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:47:41.631681 7261 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:47:41.632741 7261 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:47:41.632774 7261 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 08:47:41.632786 7261 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:47:41.632797 7261 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:47:41.632812 7261 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:47:41.632822 7261 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:47:41.634215 7261 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:46163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786092453486433-4734-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786092453486433-4734-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.159.190:33213
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab
--principal=kudu/127.4.159.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.159.190:33213
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.159.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7261
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786092453486433-4734-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:47:41.634439 7261 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:47:41.634618 7261 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:47:41.638906 7269 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:47:41.638928 7268 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:47:41.639000 7261 server_base.cc:1034] running on GCE node
W20260807 08:47:41.638909 7271 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:47:41.639310 7261 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:47:41.639580 7261 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:47:41.640717 7261 hybrid_clock.cc:648] HybridClock initialized: now 1786092461640700 us; error 28 us; skew 500 ppm
Aug 07 08:47:41 dist-test-slave-6zbq krb5kdc[7240](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092461, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:47:41.642967 7261 init.cc:377] Logged in from keytab as kudu/127.4.159.190@KRBTEST.COM (short username kudu)
I20260807 08:47:41.643361 7261 webserver.cc:460] Webserver started at http://127.4.159.190:37835/ using document root <none> and password file <none>
I20260807 08:47:41.643544 7261 fs_manager.cc:359] Metadata directory not provided
I20260807 08:47:41.643582 7261 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:47:41.643669 7261 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:47:41.644584 7261 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786092453486433-4734-0/minicluster-data/master-0/data/instance:
uuid: "d9cac9fdcacc45b693393f03561f7b11"
format_stamp: "Formatted at 2026-08-07 08:47:41 on dist-test-slave-6zbq"
server_key: "4e34797615690002f77d1df916e495a9"
server_key_iv: "a27c7f262769b40887f80d98208436a0"
server_key_version: "encryptionkey@0"
I20260807 08:47:41.644858 7261 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786092453486433-4734-0/minicluster-data/master-0/wal/instance:
uuid: "d9cac9fdcacc45b693393f03561f7b11"
format_stamp: "Formatted at 2026-08-07 08:47:41 on dist-test-slave-6zbq"
server_key: "4e34797615690002f77d1df916e495a9"
server_key_iv: "a27c7f262769b40887f80d98208436a0"
server_key_version: "encryptionkey@0"
I20260807 08:47:41.645959 7261 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 08:47:41.646567 7287 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:41.646711 7261 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 08:47:41.646783 7261 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786092453486433-4734-0/minicluster-data/master-0/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786092453486433-4734-0/minicluster-data/master-0/wal
uuid: "d9cac9fdcacc45b693393f03561f7b11"
format_stamp: "Formatted at 2026-08-07 08:47:41 on dist-test-slave-6zbq"
server_key: "4e34797615690002f77d1df916e495a9"
server_key_iv: "a27c7f262769b40887f80d98208436a0"
server_key_version: "encryptionkey@0"
I20260807 08:47:41.646850 7261 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786092453486433-4734-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786092453486433-4734-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786092453486433-4734-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:47:41.659439 7261 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:47:41.660966 7261 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:47:41.661093 7261 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:47:41.665253 7261 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.190:33213
I20260807 08:47:41.665315 7372 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.190:33213 every 8 connection(s)
I20260807 08:47:41.665576 7261 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
I20260807 08:47:41.666169 7375 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:47:41.667408 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 7261
I20260807 08:47:41.667482 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786092453486433-4734-0/minicluster-data/master-0/wal/instance
I20260807 08:47:41.667640 4734 external_mini_cluster.cc:1428] Setting key 641e535c3f432a28dd5737d33ccebf83
I20260807 08:47:41.668488 7375 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d9cac9fdcacc45b693393f03561f7b11: Bootstrap starting.
I20260807 08:47:41.669149 7375 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d9cac9fdcacc45b693393f03561f7b11: Neither blocks nor log segments found. Creating new log.
I20260807 08:47:41.669441 7375 log.cc:824] T 00000000000000000000000000000000 P d9cac9fdcacc45b693393f03561f7b11: Log is configured to *not* fsync() on all Append() calls
I20260807 08:47:41.670042 7375 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d9cac9fdcacc45b693393f03561f7b11: No bootstrap required, opened a new log
I20260807 08:47:41.671803 7375 raft_consensus.cc:348] T 00000000000000000000000000000000 P d9cac9fdcacc45b693393f03561f7b11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9cac9fdcacc45b693393f03561f7b11" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 33213 } }
I20260807 08:47:41.671939 7375 raft_consensus.cc:374] T 00000000000000000000000000000000 P d9cac9fdcacc45b693393f03561f7b11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:47:41.671972 7375 raft_consensus.cc:729] T 00000000000000000000000000000000 P d9cac9fdcacc45b693393f03561f7b11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9cac9fdcacc45b693393f03561f7b11, State: Initialized, Role: FOLLOWER
I20260807 08:47:41.672092 7375 consensus_queue.cc:260] T 00000000000000000000000000000000 P d9cac9fdcacc45b693393f03561f7b11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9cac9fdcacc45b693393f03561f7b11" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 33213 } }
I20260807 08:47:41.672169 7375 raft_consensus.cc:388] T 00000000000000000000000000000000 P d9cac9fdcacc45b693393f03561f7b11 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:47:41.672200 7375 raft_consensus.cc:482] T 00000000000000000000000000000000 P d9cac9fdcacc45b693393f03561f7b11 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:47:41.672247 7375 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d9cac9fdcacc45b693393f03561f7b11 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:47:41.672714 7375 raft_consensus.cc:504] T 00000000000000000000000000000000 P d9cac9fdcacc45b693393f03561f7b11 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9cac9fdcacc45b693393f03561f7b11" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 33213 } }
I20260807 08:47:41.672808 7375 leader_election.cc:302] T 00000000000000000000000000000000 P d9cac9fdcacc45b693393f03561f7b11 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d9cac9fdcacc45b693393f03561f7b11; no voters:
I20260807 08:47:41.672971 7375 leader_election.cc:288] T 00000000000000000000000000000000 P d9cac9fdcacc45b693393f03561f7b11 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:47:41.673007 7382 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d9cac9fdcacc45b693393f03561f7b11 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:47:41.673144 7382 raft_consensus.cc:686] T 00000000000000000000000000000000 P d9cac9fdcacc45b693393f03561f7b11 [term 1 LEADER]: Becoming Leader. State: Replica: d9cac9fdcacc45b693393f03561f7b11, State: Running, Role: LEADER
I20260807 08:47:41.673218 7375 sys_catalog.cc:564] T 00000000000000000000000000000000 P d9cac9fdcacc45b693393f03561f7b11 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:47:41.673238 7382 consensus_queue.cc:237] T 00000000000000000000000000000000 P d9cac9fdcacc45b693393f03561f7b11 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9cac9fdcacc45b693393f03561f7b11" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 33213 } }
Aug 07 08:47:41 dist-test-slave-6zbq krb5kdc[7240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092461, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.159.190@KRBTEST.COM
I20260807 08:47:41.673540 7384 sys_catalog.cc:455] T 00000000000000000000000000000000 P d9cac9fdcacc45b693393f03561f7b11 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d9cac9fdcacc45b693393f03561f7b11" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9cac9fdcacc45b693393f03561f7b11" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 33213 } } }
I20260807 08:47:41.674393 7384 sys_catalog.cc:458] T 00000000000000000000000000000000 P d9cac9fdcacc45b693393f03561f7b11 [sys.catalog]: This master's current role is: LEADER
I20260807 08:47:41.673530 7382 sys_catalog.cc:455] T 00000000000000000000000000000000 P d9cac9fdcacc45b693393f03561f7b11 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d9cac9fdcacc45b693393f03561f7b11. Latest consensus state: current_term: 1 leader_uuid: "d9cac9fdcacc45b693393f03561f7b11" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9cac9fdcacc45b693393f03561f7b11" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 33213 } } }
I20260807 08:47:41.674491 7382 sys_catalog.cc:458] T 00000000000000000000000000000000 P d9cac9fdcacc45b693393f03561f7b11 [sys.catalog]: This master's current role is: LEADER
W20260807 08:47:41.675400 7403 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d9cac9fdcacc45b693393f03561f7b11: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 08:47:41.675462 7403 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 08:47:41.675514 7406 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:47:41.675676 7406 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:47:41.677112 7406 catalog_manager.cc:1347] Generated new cluster ID: 1ae141665f234ab084b4f1f3569e7046
I20260807 08:47:41.677165 7406 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:47:41.677742 7380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:41.668082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51352 (local address 127.4.159.190:33213)
0807 08:47:41.668240 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:41.668249 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:41.668305 (+ 56us) server_negotiation.cc:407] Connection header received
0807 08:47:41.668798 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:41.668807 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:41.668884 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:41.669025 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:47:41.669598 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:41.670517 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:41.671220 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:41.671615 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:41.671705 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:41.674068 (+ 2363us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:47:41.674072 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:47:41.674072 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:47:41.674084 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:47:41.676101 (+ 2017us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:41.676485 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:41.676489 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:41.676489 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:41.676585 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:41.676809 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:41.676810 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:41.676810 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:41.677087 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:47:41.677223 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:41.677450 (+ 227us) server_negotiation.cc:299] Negotiation successful
0807 08:47:41.677557 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":61,"threads_started":1}
I20260807 08:47:41.706861 7406 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:47:41.707330 7406 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:47:41.716449 7406 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d9cac9fdcacc45b693393f03561f7b11: Generated new TSK 0
I20260807 08:47:41.716602 7406 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.159.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.159.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.159.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.129.keytab.
Entry for principal kudu/127.4.159.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.129.keytab.
I20260807 08:47:41.747189 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786092453486433-4734-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786092453486433-4734-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786092453486433-4734-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786092453486433-4734-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.159.129:0
--local_ip_for_outbound_sockets=127.4.159.129
--webserver_interface=127.4.159.129
--webserver_port=0
--tserver_master_addrs=127.4.159.190:33213
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.129.keytab
--principal=kudu/127.4.159.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:46163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:47:41.824941 7413 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:47:41.825114 7413 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:47:41.825135 7413 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:47:41.826200 7413 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:47:41.826239 7413 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:47:41.826258 7413 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 08:47:41.826288 7413 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 08:47:41.826318 7413 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.159.129
I20260807 08:47:41.827720 7413 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:46163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786092453486433-4734-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786092453486433-4734-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.129.keytab
--principal=kudu/127.4.159.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.159.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786092453486433-4734-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.159.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.4.159.190:33213
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7413
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.159.129
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786092453486433-4734-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:47:41.827922 7413 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:47:41.828114 7413 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:47:41.832679 7419 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:47:41.832670 7420 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:47:41.832773 7413 server_base.cc:1034] running on GCE node
W20260807 08:47:41.832671 7422 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:47:41.833020 7413 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:47:41.833261 7413 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:47:41.834393 7413 hybrid_clock.cc:648] HybridClock initialized: now 1786092461834378 us; error 27 us; skew 500 ppm
Aug 07 08:47:41 dist-test-slave-6zbq krb5kdc[7240](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092461, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:47:41.836654 7413 init.cc:377] Logged in from keytab as kudu/127.4.159.129@KRBTEST.COM (short username kudu)
I20260807 08:47:41.837056 7413 webserver.cc:460] Webserver started at http://127.4.159.129:40469/ using document root <none> and password file <none>
I20260807 08:47:41.837257 7413 fs_manager.cc:359] Metadata directory not provided
I20260807 08:47:41.837304 7413 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:47:41.837430 7413 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:47:41.838418 7413 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786092453486433-4734-0/minicluster-data/ts-0/data/instance:
uuid: "fd480a8c763249ba8217c035bf4fc386"
format_stamp: "Formatted at 2026-08-07 08:47:41 on dist-test-slave-6zbq"
server_key: "4659e01642d188a46415411e979ac972"
server_key_iv: "8174537371de3d1ef76bd68a360fc55d"
server_key_version: "encryptionkey@0"
I20260807 08:47:41.838755 7413 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786092453486433-4734-0/minicluster-data/ts-0/wal/instance:
uuid: "fd480a8c763249ba8217c035bf4fc386"
format_stamp: "Formatted at 2026-08-07 08:47:41 on dist-test-slave-6zbq"
server_key: "4659e01642d188a46415411e979ac972"
server_key_iv: "8174537371de3d1ef76bd68a360fc55d"
server_key_version: "encryptionkey@0"
I20260807 08:47:41.839931 7413 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 08:47:41.840561 7431 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:41.840714 7413 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 08:47:41.840804 7413 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786092453486433-4734-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786092453486433-4734-0/minicluster-data/ts-0/wal
uuid: "fd480a8c763249ba8217c035bf4fc386"
format_stamp: "Formatted at 2026-08-07 08:47:41 on dist-test-slave-6zbq"
server_key: "4659e01642d188a46415411e979ac972"
server_key_iv: "8174537371de3d1ef76bd68a360fc55d"
server_key_version: "encryptionkey@0"
I20260807 08:47:41.840876 7413 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786092453486433-4734-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786092453486433-4734-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786092453486433-4734-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:47:41.853237 7413 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:47:41.854909 7413 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:47:41.855036 7413 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:47:41.855479 7413 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:47:41.855520 7413 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:41.855545 7413 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:47:41.855561 7413 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:41.861816 7413 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.129:46375
I20260807 08:47:41.861871 7595 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.129:46375 every 8 connection(s)
Aug 07 08:47:41 dist-test-slave-6zbq krb5kdc[7240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092461, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.129@KRBTEST.COM for kudu/127.4.159.190@KRBTEST.COM
I20260807 08:47:41.862171 7413 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786092453486433-4734-0/minicluster-data/ts-0/data/info.pb
I20260807 08:47:41.864718 7380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:41.855900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.129:50951 (local address 127.4.159.190:33213)
0807 08:47:41.855947 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:41.855949 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:41.856149 (+ 200us) server_negotiation.cc:407] Connection header received
0807 08:47:41.856593 (+ 444us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:41.856595 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:41.856609 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:41.856682 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:47:41.857412 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:41.858184 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:41.859609 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:41.859935 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:41.860005 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:41.862416 (+ 2411us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:47:41.862418 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:47:41.862418 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:47:41.862426 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:47:41.863492 (+ 1066us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:41.863844 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:41.863845 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:41.863846 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:41.863903 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:41.864120 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:41.864121 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:41.864121 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:41.864233 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:47:41.864289 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:41.864534 (+ 245us) server_negotiation.cc:299] Negotiation successful
0807 08:47:41.864624 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 08:47:41.864849 7442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:41.855897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:33213 (local address 127.4.159.129:50951)
0807 08:47:41.856036 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0807 08:47:41.856069 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0807 08:47:41.856500 (+ 431us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:47:41.856734 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:47:41.856738 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:47:41.857027 (+ 289us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:47:41.857309 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:41.857316 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:41.858276 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:41.858277 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:41.859499 (+ 1222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:41.859503 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:41.860041 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:41.860041 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:41.860176 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:41.860674 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:47:41.860684 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:47:41.862301 (+ 1617us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:47:41.863565 (+ 1264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:47:41.863571 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:47:41.863573 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:47:41.863779 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:47:41.863955 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:47:41.863956 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:47:41.863956 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:47:41.864059 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:47:41.864288 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:47:41.864295 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:47:41.864420 (+ 125us) client_negotiation.cc:769] Sending connection context
0807 08:47:41.864505 (+ 85us) client_negotiation.cc:240] Negotiation successful
0807 08:47:41.864657 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":80,"threads_started":1}
I20260807 08:47:41.865257 7596 heartbeater.cc:347] Connected to a master server at 127.4.159.190:33213
I20260807 08:47:41.865352 7596 heartbeater.cc:464] Registering TS with master...
I20260807 08:47:41.865516 7596 heartbeater.cc:511] Master 127.4.159.190:33213 requested a full tablet report, sending...
I20260807 08:47:41.866009 7310 ts_manager.cc:194] Registered new tserver with Master: fd480a8c763249ba8217c035bf4fc386 (127.4.159.129:46375)
I20260807 08:47:41.866681 7310 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.159.129@KRBTEST.COM'} at 127.4.159.129:50951
I20260807 08:47:41.870769 7380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:41.866187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.129:39253 (local address 127.4.159.190:33213)
0807 08:47:41.866231 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:41.866232 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:41.866367 (+ 135us) server_negotiation.cc:407] Connection header received
0807 08:47:41.866409 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:41.866410 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:41.866419 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:41.866469 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:47:41.866737 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:41.867296 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:41.867935 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:41.868136 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:41.868187 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:41.868869 (+ 682us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:47:41.868870 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:47:41.868871 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:47:41.868877 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:47:41.869795 (+ 918us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:41.870073 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:41.870074 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:41.870074 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:41.870125 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:41.870280 (+ 155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:41.870281 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:41.870281 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:41.870416 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:47:41.870482 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:41.870619 (+ 137us) server_negotiation.cc:299] Negotiation successful
0807 08:47:41.870686 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 08:47:41.870769 7605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:41.866139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:33213 (local address 127.4.159.129:39253)
0807 08:47:41.866285 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0807 08:47:41.866294 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 08:47:41.866360 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:47:41.866474 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:47:41.866474 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:47:41.866629 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:47:41.866675 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:41.866676 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:41.867380 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:41.867381 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:41.867869 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:41.867870 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:41.868190 (+ 320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:41.868191 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:41.868266 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:41.868560 (+ 294us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:47:41.868563 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:47:41.868795 (+ 232us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:47:41.869864 (+ 1069us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:47:41.869865 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:47:41.869866 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:47:41.870016 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:47:41.870177 (+ 161us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:47:41.870177 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:47:41.870178 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:47:41.870231 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:47:41.870482 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:47:41.870482 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:47:41.870526 (+ 44us) client_negotiation.cc:769] Sending connection context
0807 08:47:41.870585 (+ 59us) client_negotiation.cc:240] Negotiation successful
0807 08:47:41.870676 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":64,"threads_started":1}
I20260807 08:47:41.871183 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 7413
I20260807 08:47:41.871261 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786092453486433-4734-0/minicluster-data/ts-0/wal/instance
I20260807 08:47:41.871418 4734 external_mini_cluster.cc:1428] Setting key 6c73ca3c68fba28e4e3f6b34bdb0e358
WARNING: no policy specified for kudu/127.4.159.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.159.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.159.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.130.keytab.
Entry for principal kudu/127.4.159.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.130.keytab.
I20260807 08:47:41.892769 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786092453486433-4734-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786092453486433-4734-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786092453486433-4734-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786092453486433-4734-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.159.130:0
--local_ip_for_outbound_sockets=127.4.159.130
--webserver_interface=127.4.159.130
--webserver_port=0
--tserver_master_addrs=127.4.159.190:33213
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.130.keytab
--principal=kudu/127.4.159.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:46163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:47:41.969170 7608 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:47:41.969347 7608 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:47:41.969370 7608 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:47:41.970460 7608 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:47:41.970495 7608 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:47:41.970517 7608 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 08:47:41.970539 7608 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 08:47:41.970569 7608 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.159.130
I20260807 08:47:41.971943 7608 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:46163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786092453486433-4734-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786092453486433-4734-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.130.keytab
--principal=kudu/127.4.159.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.159.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786092453486433-4734-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.159.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.4.159.190:33213
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7608
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.159.130
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786092453486433-4734-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:47:41.972127 7608 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:47:41.972316 7608 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:47:41.976854 7620 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:47:41.976859 7615 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:47:41.977044 7608 server_base.cc:1034] running on GCE node
W20260807 08:47:41.976861 7618 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:47:41.977248 7608 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:47:41.977464 7608 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:47:41.978598 7608 hybrid_clock.cc:648] HybridClock initialized: now 1786092461978582 us; error 28 us; skew 500 ppm
Aug 07 08:47:41 dist-test-slave-6zbq krb5kdc[7240](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092461, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:47:41.980706 7608 init.cc:377] Logged in from keytab as kudu/127.4.159.130@KRBTEST.COM (short username kudu)
I20260807 08:47:41.981110 7608 webserver.cc:460] Webserver started at http://127.4.159.130:38633/ using document root <none> and password file <none>
I20260807 08:47:41.981312 7608 fs_manager.cc:359] Metadata directory not provided
I20260807 08:47:41.981357 7608 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:47:41.981472 7608 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:47:41.982411 7608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786092453486433-4734-0/minicluster-data/ts-1/data/instance:
uuid: "56aa53ddfa8d405db14e40d13c45f344"
format_stamp: "Formatted at 2026-08-07 08:47:41 on dist-test-slave-6zbq"
server_key: "17a550e897ee270a7d61175676dabc36"
server_key_iv: "b3f081fd631037feae6197f70050bb03"
server_key_version: "encryptionkey@0"
I20260807 08:47:41.982724 7608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786092453486433-4734-0/minicluster-data/ts-1/wal/instance:
uuid: "56aa53ddfa8d405db14e40d13c45f344"
format_stamp: "Formatted at 2026-08-07 08:47:41 on dist-test-slave-6zbq"
server_key: "17a550e897ee270a7d61175676dabc36"
server_key_iv: "b3f081fd631037feae6197f70050bb03"
server_key_version: "encryptionkey@0"
I20260807 08:47:41.983855 7608 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 08:47:41.984455 7629 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:41.984601 7608 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 08:47:41.984668 7608 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786092453486433-4734-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786092453486433-4734-0/minicluster-data/ts-1/wal
uuid: "56aa53ddfa8d405db14e40d13c45f344"
format_stamp: "Formatted at 2026-08-07 08:47:41 on dist-test-slave-6zbq"
server_key: "17a550e897ee270a7d61175676dabc36"
server_key_iv: "b3f081fd631037feae6197f70050bb03"
server_key_version: "encryptionkey@0"
I20260807 08:47:41.984730 7608 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786092453486433-4734-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786092453486433-4734-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786092453486433-4734-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:47:41.996542 7608 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:47:41.997993 7608 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:47:41.998119 7608 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:47:41.998577 7608 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:47:41.998615 7608 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:41.998642 7608 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:47:41.998670 7608 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 08:47:41 dist-test-slave-6zbq krb5kdc[7240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092461, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.130@KRBTEST.COM for kudu/127.4.159.190@KRBTEST.COM
I20260807 08:47:42.005276 7608 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.130:35393
I20260807 08:47:42.005365 7802 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.130:35393 every 8 connection(s)
I20260807 08:47:42.005625 7608 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786092453486433-4734-0/minicluster-data/ts-1/data/info.pb
I20260807 08:47:42.006520 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 7608
I20260807 08:47:42.006613 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786092453486433-4734-0/minicluster-data/ts-1/wal/instance
I20260807 08:47:42.006788 4734 external_mini_cluster.cc:1428] Setting key 3d8f7ac2bdc40d20574b3d7c5cf0961c
I20260807 08:47:42.007683 7380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:41.998912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.130:53099 (local address 127.4.159.190:33213)
0807 08:47:41.998974 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:41.998976 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:41.999143 (+ 167us) server_negotiation.cc:407] Connection header received
0807 08:47:41.999525 (+ 382us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:41.999527 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:41.999541 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:41.999604 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:47:42.000196 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:42.000730 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:42.001702 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:42.001932 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:42.001986 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:42.004682 (+ 2696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:47:42.004684 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:47:42.004685 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:47:42.004693 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:47:42.006128 (+ 1435us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:42.006579 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:42.006581 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:42.006581 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:42.006646 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:42.006908 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:42.006910 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:42.006910 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:42.007047 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:47:42.007118 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:42.007552 (+ 434us) server_negotiation.cc:299] Negotiation successful
0807 08:47:42.007620 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260807 08:47:42.007856 7641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:41.998894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:33213 (local address 127.4.159.130:53099)
0807 08:47:41.999034 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0807 08:47:41.999069 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0807 08:47:41.999443 (+ 374us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:47:41.999663 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:47:41.999666 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:47:41.999916 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:47:42.000111 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:42.000116 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:42.000807 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:42.000808 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:42.001617 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:42.001619 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:42.002011 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:42.002011 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:42.002141 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:42.002619 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:47:42.002629 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:47:42.004605 (+ 1976us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:47:42.006206 (+ 1601us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:47:42.006211 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:47:42.006213 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:47:42.006507 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:47:42.006706 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:47:42.006707 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:47:42.006707 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:47:42.006839 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:47:42.007301 (+ 462us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:47:42.007306 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:47:42.007419 (+ 113us) client_negotiation.cc:769] Sending connection context
0807 08:47:42.007529 (+ 110us) client_negotiation.cc:240] Negotiation successful
0807 08:47:42.007674 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":79,"threads_started":1}
I20260807 08:47:42.008339 7803 heartbeater.cc:347] Connected to a master server at 127.4.159.190:33213
I20260807 08:47:42.008421 7803 heartbeater.cc:464] Registering TS with master...
I20260807 08:47:42.008576 7803 heartbeater.cc:511] Master 127.4.159.190:33213 requested a full tablet report, sending...
I20260807 08:47:42.008909 7311 ts_manager.cc:194] Registered new tserver with Master: 56aa53ddfa8d405db14e40d13c45f344 (127.4.159.130:35393)
I20260807 08:47:42.009385 7311 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.159.130@KRBTEST.COM'} at 127.4.159.130:53099
I20260807 08:47:42.014139 7814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:42.009243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:33213 (local address 127.4.159.130:50989)
0807 08:47:42.009352 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0807 08:47:42.009361 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 08:47:42.009404 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:47:42.009613 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:47:42.009613 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:47:42.009781 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:47:42.009833 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:42.009834 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:42.010452 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:42.010453 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:42.010956 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:42.010957 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:42.011270 (+ 313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:42.011271 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:42.011342 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:42.011675 (+ 333us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:47:42.011679 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:47:42.011935 (+ 256us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:47:42.013252 (+ 1317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:47:42.013253 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:47:42.013253 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:47:42.013403 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:47:42.013561 (+ 158us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:47:42.013562 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:47:42.013562 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:47:42.013616 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:47:42.013856 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:47:42.013856 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:47:42.013899 (+ 43us) client_negotiation.cc:769] Sending connection context
0807 08:47:42.013958 (+ 59us) client_negotiation.cc:240] Negotiation successful
0807 08:47:42.014047 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":87,"thread_start_us":61,"threads_started":1}
I20260807 08:47:42.014143 7380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:42.009293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.130:50989 (local address 127.4.159.190:33213)
0807 08:47:42.009517 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:42.009518 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:42.009527 (+ 9us) server_negotiation.cc:407] Connection header received
0807 08:47:42.009545 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:42.009545 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:42.009554 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:42.009607 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:47:42.009904 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:42.010383 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:42.011023 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:42.011216 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:42.011266 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:42.012007 (+ 741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:47:42.012008 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:47:42.012009 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:47:42.012015 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:47:42.013182 (+ 1167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:42.013460 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:42.013461 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:42.013461 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:42.013512 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:42.013663 (+ 151us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:42.013665 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:42.013665 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:42.013799 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:47:42.013854 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:42.013986 (+ 132us) server_negotiation.cc:299] Negotiation successful
0807 08:47:42.014063 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":205}
WARNING: no policy specified for kudu/127.4.159.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.159.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.159.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.131.keytab.
Entry for principal kudu/127.4.159.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.131.keytab.
I20260807 08:47:42.029361 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786092453486433-4734-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786092453486433-4734-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786092453486433-4734-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786092453486433-4734-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.159.131:0
--local_ip_for_outbound_sockets=127.4.159.131
--webserver_interface=127.4.159.131
--webserver_port=0
--tserver_master_addrs=127.4.159.190:33213
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.131.keytab
--principal=kudu/127.4.159.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:46163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:47:42.107491 7817 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:47:42.107671 7817 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:47:42.107695 7817 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:47:42.108834 7817 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:47:42.108872 7817 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:47:42.108892 7817 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 08:47:42.108908 7817 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 08:47:42.108923 7817 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.159.131
I20260807 08:47:42.110330 7817 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:46163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786092453486433-4734-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786092453486433-4734-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.131.keytab
--principal=kudu/127.4.159.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.159.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786092453486433-4734-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.159.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.4.159.190:33213
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7817
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.159.131
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786092453486433-4734-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:47:42.110532 7817 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:47:42.110699 7817 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:47:42.115152 7822 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:47:42.115262 7817 server_base.cc:1034] running on GCE node
W20260807 08:47:42.115134 7827 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:47:42.115140 7823 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:47:42.115517 7817 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:47:42.115741 7817 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:47:42.116873 7817 hybrid_clock.cc:648] HybridClock initialized: now 1786092462116862 us; error 31 us; skew 500 ppm
Aug 07 08:47:42 dist-test-slave-6zbq krb5kdc[7240](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092462, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:47:42.119132 7817 init.cc:377] Logged in from keytab as kudu/127.4.159.131@KRBTEST.COM (short username kudu)
I20260807 08:47:42.119505 7817 webserver.cc:460] Webserver started at http://127.4.159.131:42285/ using document root <none> and password file <none>
I20260807 08:47:42.119683 7817 fs_manager.cc:359] Metadata directory not provided
I20260807 08:47:42.119727 7817 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:47:42.119846 7817 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:47:42.120764 7817 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786092453486433-4734-0/minicluster-data/ts-2/data/instance:
uuid: "0f212e32e8044e8c9eb244945ad1c908"
format_stamp: "Formatted at 2026-08-07 08:47:42 on dist-test-slave-6zbq"
server_key: "63cfcb3e52ab19096efc659673d1ac2a"
server_key_iv: "0c0b6e4f53710815b9250a7837ea36c0"
server_key_version: "encryptionkey@0"
I20260807 08:47:42.121073 7817 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786092453486433-4734-0/minicluster-data/ts-2/wal/instance:
uuid: "0f212e32e8044e8c9eb244945ad1c908"
format_stamp: "Formatted at 2026-08-07 08:47:42 on dist-test-slave-6zbq"
server_key: "63cfcb3e52ab19096efc659673d1ac2a"
server_key_iv: "0c0b6e4f53710815b9250a7837ea36c0"
server_key_version: "encryptionkey@0"
I20260807 08:47:42.122233 7817 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 08:47:42.122896 7835 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:42.123039 7817 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:42.123117 7817 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786092453486433-4734-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786092453486433-4734-0/minicluster-data/ts-2/wal
uuid: "0f212e32e8044e8c9eb244945ad1c908"
format_stamp: "Formatted at 2026-08-07 08:47:42 on dist-test-slave-6zbq"
server_key: "63cfcb3e52ab19096efc659673d1ac2a"
server_key_iv: "0c0b6e4f53710815b9250a7837ea36c0"
server_key_version: "encryptionkey@0"
I20260807 08:47:42.123178 7817 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786092453486433-4734-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786092453486433-4734-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786092453486433-4734-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:47:42.143388 7817 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:47:42.145329 7817 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:47:42.145452 7817 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:47:42.145879 7817 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:47:42.145920 7817 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:42.145948 7817 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:47:42.145972 7817 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 08:47:42 dist-test-slave-6zbq krb5kdc[7240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092462, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.131@KRBTEST.COM for kudu/127.4.159.190@KRBTEST.COM
I20260807 08:47:42.152635 7817 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.131:45951
I20260807 08:47:42.152699 7995 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.131:45951 every 8 connection(s)
I20260807 08:47:42.153025 7817 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786092453486433-4734-0/minicluster-data/ts-2/data/info.pb
I20260807 08:47:42.153507 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 7817
I20260807 08:47:42.153602 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786092453486433-4734-0/minicluster-data/ts-2/wal/instance
I20260807 08:47:42.153776 4734 external_mini_cluster.cc:1428] Setting key 49e5e1147881332344d64fbc59fb8600
I20260807 08:47:42.155611 7380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:42.146361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.131:42953 (local address 127.4.159.190:33213)
0807 08:47:42.146403 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:42.146405 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:42.146607 (+ 202us) server_negotiation.cc:407] Connection header received
0807 08:47:42.147055 (+ 448us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:42.147057 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:42.147070 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:42.147121 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:47:42.147727 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:42.148474 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:42.149952 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:42.150240 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:42.150313 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:42.152820 (+ 2507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:47:42.152822 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:47:42.152822 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:47:42.152830 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:47:42.154625 (+ 1795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:42.154965 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:42.154966 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:42.154966 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:42.155015 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:42.155174 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:42.155175 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:42.155175 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:42.155265 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:47:42.155312 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:42.155497 (+ 185us) server_negotiation.cc:299] Negotiation successful
0807 08:47:42.155552 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260807 08:47:42.155783 7849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:42.146326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:33213 (local address 127.4.159.131:42953)
0807 08:47:42.146504 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0807 08:47:42.146536 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0807 08:47:42.146963 (+ 427us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:47:42.147185 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:47:42.147189 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:47:42.147464 (+ 275us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:47:42.147637 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:42.147642 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:42.148587 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:42.148588 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:42.149857 (+ 1269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:42.149861 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:42.150357 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:42.150358 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:42.150500 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:42.151004 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:47:42.151014 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:47:42.152685 (+ 1671us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:47:42.154695 (+ 2010us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:47:42.154701 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:47:42.154703 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:47:42.154908 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:47:42.155053 (+ 145us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:47:42.155054 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:47:42.155054 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:47:42.155136 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:47:42.155307 (+ 171us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:47:42.155310 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:47:42.155414 (+ 104us) client_negotiation.cc:769] Sending connection context
0807 08:47:42.155486 (+ 72us) client_negotiation.cc:240] Negotiation successful
0807 08:47:42.155595 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":79,"threads_started":1}
I20260807 08:47:42.156184 7996 heartbeater.cc:347] Connected to a master server at 127.4.159.190:33213
I20260807 08:47:42.156267 7996 heartbeater.cc:464] Registering TS with master...
I20260807 08:47:42.156440 7996 heartbeater.cc:511] Master 127.4.159.190:33213 requested a full tablet report, sending...
I20260807 08:47:42.156785 7310 ts_manager.cc:194] Registered new tserver with Master: 0f212e32e8044e8c9eb244945ad1c908 (127.4.159.131:45951)
I20260807 08:47:42.157223 7310 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.159.131@KRBTEST.COM'} at 127.4.159.131:42953
I20260807 08:47:42.161634 7380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:42.157149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.131:53929 (local address 127.4.159.190:33213)
0807 08:47:42.157184 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:42.157186 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:42.157301 (+ 115us) server_negotiation.cc:407] Connection header received
0807 08:47:42.157366 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:42.157367 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:42.157378 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:42.157431 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:47:42.157717 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:42.158162 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:42.158869 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:42.159061 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:42.159112 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:42.159806 (+ 694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:47:42.159807 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:47:42.159807 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:47:42.159814 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:47:42.160718 (+ 904us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:42.160990 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:42.160991 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:42.160992 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:42.161042 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:42.161206 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:42.161206 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:42.161207 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:42.161306 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:47:42.161357 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:42.161489 (+ 132us) server_negotiation.cc:299] Negotiation successful
0807 08:47:42.161550 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260807 08:47:42.161640 8006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:42.157091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:33213 (local address 127.4.159.131:53929)
0807 08:47:42.157241 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0807 08:47:42.157249 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0807 08:47:42.157303 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:47:42.157448 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:47:42.157449 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:47:42.157606 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:47:42.157654 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:42.157655 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:42.158241 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:42.158242 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:42.158800 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:42.158801 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:42.159116 (+ 315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:42.159117 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:42.159193 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:42.159490 (+ 297us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:47:42.159494 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:47:42.159728 (+ 234us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:47:42.160784 (+ 1056us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:47:42.160785 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:47:42.160785 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:47:42.160933 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:47:42.161102 (+ 169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:47:42.161102 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:47:42.161103 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:47:42.161157 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:47:42.161356 (+ 199us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:47:42.161357 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:47:42.161400 (+ 43us) client_negotiation.cc:769] Sending connection context
0807 08:47:42.161459 (+ 59us) client_negotiation.cc:240] Negotiation successful
0807 08:47:42.161550 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":68,"threads_started":1}
I20260807 08:47:42.165457 4734 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 08:47:42 dist-test-slave-6zbq krb5kdc[7240](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092462, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 08:47:42 dist-test-slave-6zbq krb5kdc[7240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092462, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.159.190@KRBTEST.COM
I20260807 08:47:42.186897 7380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:42.181174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51364 (local address 127.4.159.190:33213)
0807 08:47:42.181222 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:42.181223 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:42.181231 (+ 8us) server_negotiation.cc:407] Connection header received
0807 08:47:42.181265 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:42.181266 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:42.181273 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:42.181308 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:47:42.181744 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:42.182176 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:42.182847 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:42.183015 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:42.183061 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:42.184816 (+ 1755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:47:42.184817 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:47:42.184817 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:47:42.184825 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:47:42.185809 (+ 984us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:42.186169 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:42.186170 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:42.186170 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:42.186255 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:42.186487 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:42.186487 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:42.186488 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:42.186609 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:47:42.186683 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:42.186787 (+ 104us) server_negotiation.cc:299] Negotiation successful
0807 08:47:42.186842 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260807 08:47:42.188150 7310 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51364:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 08:47:42.189581 7310 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:47:42.195081 8024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:42.191385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.131:45951 (local address 127.0.0.1:42090)
0807 08:47:42.191651 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0807 08:47:42.191690 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 08:47:42.191760 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:47:42.191953 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:47:42.191956 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:47:42.191959 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:47:42.192019 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:42.192021 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:42.192939 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:42.192940 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:42.193947 (+ 1007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:42.193949 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:42.194850 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:42.194851 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:42.194905 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:42.194913 (+ 8us) client_negotiation.cc:769] Sending connection context
0807 08:47:42.194964 (+ 51us) client_negotiation.cc:240] Negotiation successful
0807 08:47:42.195013 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":48,"threads_started":1}
I20260807 08:47:42.195107 8022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:42.191456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42090 (local address 127.4.159.131:45951)
0807 08:47:42.191604 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:42.191608 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:42.191746 (+ 138us) server_negotiation.cc:407] Connection header received
0807 08:47:42.191845 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:42.191847 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:42.191862 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:42.191938 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:47:42.192104 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:42.192866 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:42.194041 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:42.194737 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:42.194768 (+ 31us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:42.194805 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:42.194996 (+ 191us) server_negotiation.cc:299] Negotiation successful
0807 08:47:42.195029 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":80,"threads_started":1}
I20260807 08:47:42.195394 8019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:42.191408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.129:46375 (local address 127.0.0.1:57780)
0807 08:47:42.191698 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0807 08:47:42.191707 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 08:47:42.191762 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:47:42.192175 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:47:42.192176 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:47:42.192178 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:47:42.192207 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:42.192208 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:42.193443 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:42.193444 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:42.194235 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:42.194238 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:42.195172 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:42.195174 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:42.195237 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:42.195240 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 08:47:42.195288 (+ 48us) client_negotiation.cc:240] Negotiation successful
0807 08:47:42.195326 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":52,"threads_started":1}
I20260807 08:47:42.195647 8021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:42.191564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.130:35393 (local address 127.0.0.1:55864)
0807 08:47:42.191865 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0807 08:47:42.191873 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0807 08:47:42.191911 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:47:42.192068 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:47:42.192069 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:47:42.192071 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:47:42.192123 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:42.192124 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:42.193396 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:42.193397 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:42.194520 (+ 1123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:42.194523 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:42.195445 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:42.195446 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:42.195503 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:42.195507 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 08:47:42.195551 (+ 44us) client_negotiation.cc:240] Negotiation successful
0807 08:47:42.195585 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"mutex_wait_us":1,"thread_start_us":41,"threads_started":1}
I20260807 08:47:42.195694 8025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:42.191529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57780 (local address 127.4.159.129:46375)
0807 08:47:42.191675 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:42.191679 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:42.191768 (+ 89us) server_negotiation.cc:407] Connection header received
0807 08:47:42.192022 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:42.192025 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:42.192038 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:42.192107 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:47:42.192280 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:42.193384 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:42.194323 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:42.195019 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:42.195086 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:42.195136 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:42.195564 (+ 428us) server_negotiation.cc:299] Negotiation successful
0807 08:47:42.195600 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":85,"threads_started":1}
I20260807 08:47:42.195811 7905 tablet_service.cc:1467] Processing CreateTablet for tablet f107b3c597cb4bdca57052aa5f9930bd (DEFAULT_TABLE table=test-table [id=d114a1b1fd124ae596ed4ab61ab93df9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:47:42.195945 8028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:42.191736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55864 (local address 127.4.159.130:35393)
0807 08:47:42.191918 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:42.191922 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:42.191932 (+ 10us) server_negotiation.cc:407] Connection header received
0807 08:47:42.191966 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:42.191968 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:42.191982 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:42.192063 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:47:42.192203 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:42.193306 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:42.194603 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:42.195289 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:42.195357 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:42.195409 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:42.195806 (+ 397us) server_negotiation.cc:299] Negotiation successful
0807 08:47:42.195852 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":81,"threads_started":1}
I20260807 08:47:42.196151 7905 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f107b3c597cb4bdca57052aa5f9930bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:47:42.196239 7505 tablet_service.cc:1467] Processing CreateTablet for tablet f107b3c597cb4bdca57052aa5f9930bd (DEFAULT_TABLE table=test-table [id=d114a1b1fd124ae596ed4ab61ab93df9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:47:42.196564 7505 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f107b3c597cb4bdca57052aa5f9930bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:47:42.196604 7711 tablet_service.cc:1467] Processing CreateTablet for tablet f107b3c597cb4bdca57052aa5f9930bd (DEFAULT_TABLE table=test-table [id=d114a1b1fd124ae596ed4ab61ab93df9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:47:42.196905 7711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f107b3c597cb4bdca57052aa5f9930bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:47:42.198894 8029 tablet_bootstrap.cc:492] T f107b3c597cb4bdca57052aa5f9930bd P 0f212e32e8044e8c9eb244945ad1c908: Bootstrap starting.
I20260807 08:47:42.199632 8029 tablet_bootstrap.cc:654] T f107b3c597cb4bdca57052aa5f9930bd P 0f212e32e8044e8c9eb244945ad1c908: Neither blocks nor log segments found. Creating new log.
I20260807 08:47:42.199900 8029 log.cc:824] T f107b3c597cb4bdca57052aa5f9930bd P 0f212e32e8044e8c9eb244945ad1c908: Log is configured to *not* fsync() on all Append() calls
I20260807 08:47:42.200346 8030 tablet_bootstrap.cc:492] T f107b3c597cb4bdca57052aa5f9930bd P fd480a8c763249ba8217c035bf4fc386: Bootstrap starting.
I20260807 08:47:42.200412 8031 tablet_bootstrap.cc:492] T f107b3c597cb4bdca57052aa5f9930bd P 56aa53ddfa8d405db14e40d13c45f344: Bootstrap starting.
I20260807 08:47:42.200603 8029 tablet_bootstrap.cc:492] T f107b3c597cb4bdca57052aa5f9930bd P 0f212e32e8044e8c9eb244945ad1c908: No bootstrap required, opened a new log
I20260807 08:47:42.200666 8029 ts_tablet_manager.cc:1397] T f107b3c597cb4bdca57052aa5f9930bd P 0f212e32e8044e8c9eb244945ad1c908: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 08:47:42.200955 8030 tablet_bootstrap.cc:654] T f107b3c597cb4bdca57052aa5f9930bd P fd480a8c763249ba8217c035bf4fc386: Neither blocks nor log segments found. Creating new log.
I20260807 08:47:42.200955 8031 tablet_bootstrap.cc:654] T f107b3c597cb4bdca57052aa5f9930bd P 56aa53ddfa8d405db14e40d13c45f344: Neither blocks nor log segments found. Creating new log.
I20260807 08:47:42.201300 8031 log.cc:824] T f107b3c597cb4bdca57052aa5f9930bd P 56aa53ddfa8d405db14e40d13c45f344: Log is configured to *not* fsync() on all Append() calls
I20260807 08:47:42.201306 8030 log.cc:824] T f107b3c597cb4bdca57052aa5f9930bd P fd480a8c763249ba8217c035bf4fc386: Log is configured to *not* fsync() on all Append() calls
I20260807 08:47:42.202013 8031 tablet_bootstrap.cc:492] T f107b3c597cb4bdca57052aa5f9930bd P 56aa53ddfa8d405db14e40d13c45f344: No bootstrap required, opened a new log
I20260807 08:47:42.202073 8030 tablet_bootstrap.cc:492] T f107b3c597cb4bdca57052aa5f9930bd P fd480a8c763249ba8217c035bf4fc386: No bootstrap required, opened a new log
I20260807 08:47:42.202083 8031 ts_tablet_manager.cc:1397] T f107b3c597cb4bdca57052aa5f9930bd P 56aa53ddfa8d405db14e40d13c45f344: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 08:47:42.202136 8030 ts_tablet_manager.cc:1397] T f107b3c597cb4bdca57052aa5f9930bd P fd480a8c763249ba8217c035bf4fc386: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 08:47:42.202499 8029 raft_consensus.cc:348] T f107b3c597cb4bdca57052aa5f9930bd P 0f212e32e8044e8c9eb244945ad1c908 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f212e32e8044e8c9eb244945ad1c908" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 45951 } } peers { permanent_uuid: "fd480a8c763249ba8217c035bf4fc386" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 46375 } } peers { permanent_uuid: "56aa53ddfa8d405db14e40d13c45f344" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 35393 } }
I20260807 08:47:42.202666 8029 raft_consensus.cc:374] T f107b3c597cb4bdca57052aa5f9930bd P 0f212e32e8044e8c9eb244945ad1c908 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:47:42.202708 8029 raft_consensus.cc:729] T f107b3c597cb4bdca57052aa5f9930bd P 0f212e32e8044e8c9eb244945ad1c908 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f212e32e8044e8c9eb244945ad1c908, State: Initialized, Role: FOLLOWER
I20260807 08:47:42.202845 8029 consensus_queue.cc:260] T f107b3c597cb4bdca57052aa5f9930bd P 0f212e32e8044e8c9eb244945ad1c908 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f212e32e8044e8c9eb244945ad1c908" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 45951 } } peers { permanent_uuid: "fd480a8c763249ba8217c035bf4fc386" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 46375 } } peers { permanent_uuid: "56aa53ddfa8d405db14e40d13c45f344" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 35393 } }
I20260807 08:47:42.203116 8029 ts_tablet_manager.cc:1428] T f107b3c597cb4bdca57052aa5f9930bd P 0f212e32e8044e8c9eb244945ad1c908: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260807 08:47:42.203140 7996 heartbeater.cc:503] Master 127.4.159.190:33213 was elected leader, sending a full tablet report...
I20260807 08:47:42.203771 8030 raft_consensus.cc:348] T f107b3c597cb4bdca57052aa5f9930bd P fd480a8c763249ba8217c035bf4fc386 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f212e32e8044e8c9eb244945ad1c908" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 45951 } } peers { permanent_uuid: "fd480a8c763249ba8217c035bf4fc386" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 46375 } } peers { permanent_uuid: "56aa53ddfa8d405db14e40d13c45f344" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 35393 } }
I20260807 08:47:42.203933 8030 raft_consensus.cc:374] T f107b3c597cb4bdca57052aa5f9930bd P fd480a8c763249ba8217c035bf4fc386 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:47:42.203976 8030 raft_consensus.cc:729] T f107b3c597cb4bdca57052aa5f9930bd P fd480a8c763249ba8217c035bf4fc386 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd480a8c763249ba8217c035bf4fc386, State: Initialized, Role: FOLLOWER
I20260807 08:47:42.203928 8031 raft_consensus.cc:348] T f107b3c597cb4bdca57052aa5f9930bd P 56aa53ddfa8d405db14e40d13c45f344 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f212e32e8044e8c9eb244945ad1c908" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 45951 } } peers { permanent_uuid: "fd480a8c763249ba8217c035bf4fc386" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 46375 } } peers { permanent_uuid: "56aa53ddfa8d405db14e40d13c45f344" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 35393 } }
I20260807 08:47:42.204087 8031 raft_consensus.cc:374] T f107b3c597cb4bdca57052aa5f9930bd P 56aa53ddfa8d405db14e40d13c45f344 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:47:42.204083 8030 consensus_queue.cc:260] T f107b3c597cb4bdca57052aa5f9930bd P fd480a8c763249ba8217c035bf4fc386 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f212e32e8044e8c9eb244945ad1c908" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 45951 } } peers { permanent_uuid: "fd480a8c763249ba8217c035bf4fc386" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 46375 } } peers { permanent_uuid: "56aa53ddfa8d405db14e40d13c45f344" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 35393 } }
I20260807 08:47:42.204133 8031 raft_consensus.cc:729] T f107b3c597cb4bdca57052aa5f9930bd P 56aa53ddfa8d405db14e40d13c45f344 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56aa53ddfa8d405db14e40d13c45f344, State: Initialized, Role: FOLLOWER
I20260807 08:47:42.204237 8031 consensus_queue.cc:260] T f107b3c597cb4bdca57052aa5f9930bd P 56aa53ddfa8d405db14e40d13c45f344 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f212e32e8044e8c9eb244945ad1c908" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 45951 } } peers { permanent_uuid: "fd480a8c763249ba8217c035bf4fc386" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 46375 } } peers { permanent_uuid: "56aa53ddfa8d405db14e40d13c45f344" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 35393 } }
I20260807 08:47:42.204335 8030 ts_tablet_manager.cc:1428] T f107b3c597cb4bdca57052aa5f9930bd P fd480a8c763249ba8217c035bf4fc386: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260807 08:47:42.204461 7596 heartbeater.cc:503] Master 127.4.159.190:33213 was elected leader, sending a full tablet report...
I20260807 08:47:42.204494 8031 ts_tablet_manager.cc:1428] T f107b3c597cb4bdca57052aa5f9930bd P 56aa53ddfa8d405db14e40d13c45f344: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260807 08:47:42.204514 7803 heartbeater.cc:503] Master 127.4.159.190:33213 was elected leader, sending a full tablet report...
I20260807 08:47:42.220600 8037 raft_consensus.cc:482] T f107b3c597cb4bdca57052aa5f9930bd P 56aa53ddfa8d405db14e40d13c45f344 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:47:42.220706 8037 raft_consensus.cc:504] T f107b3c597cb4bdca57052aa5f9930bd P 56aa53ddfa8d405db14e40d13c45f344 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f212e32e8044e8c9eb244945ad1c908" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 45951 } } peers { permanent_uuid: "fd480a8c763249ba8217c035bf4fc386" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 46375 } } peers { permanent_uuid: "56aa53ddfa8d405db14e40d13c45f344" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 35393 } }
I20260807 08:47:42.221002 8037 leader_election.cc:288] T f107b3c597cb4bdca57052aa5f9930bd P 56aa53ddfa8d405db14e40d13c45f344 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0f212e32e8044e8c9eb244945ad1c908 (127.4.159.131:45951), fd480a8c763249ba8217c035bf4fc386 (127.4.159.129:46375)
I20260807 08:47:42.224453 8025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:42.221239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.130:47341 (local address 127.4.159.129:46375)
0807 08:47:42.221273 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:42.221275 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:42.221405 (+ 130us) server_negotiation.cc:407] Connection header received
0807 08:47:42.221442 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:42.221442 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:42.221452 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:42.221494 (+ 42us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:47:42.221614 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:42.222283 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:42.223582 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:42.224129 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:42.224195 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:42.224202 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:42.224353 (+ 151us) server_negotiation.cc:299] Negotiation successful
0807 08:47:42.224388 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 08:47:42.224453 8022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:42.221229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.130:59911 (local address 127.4.159.131:45951)
0807 08:47:42.221291 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:42.221293 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:42.221300 (+ 7us) server_negotiation.cc:407] Connection header received
0807 08:47:42.221321 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:42.221322 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:42.221330 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:42.221373 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:47:42.221501 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:42.222213 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:42.223582 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:42.224132 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:42.224195 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:42.224202 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:42.224369 (+ 167us) server_negotiation.cc:299] Negotiation successful
0807 08:47:42.224398 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260807 08:47:42.224486 7641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:42.221150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.131:45951 (local address 127.4.159.130:59911)
0807 08:47:42.221201 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0807 08:47:42.221214 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:47:42.221256 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:47:42.221381 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:47:42.221382 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:47:42.221383 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:47:42.221448 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:42.221450 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:42.222278 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:42.222279 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:42.223496 (+ 1217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:42.223499 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:42.224207 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:42.224208 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:42.224305 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:42.224307 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 08:47:42.224361 (+ 54us) client_negotiation.cc:240] Negotiation successful
0807 08:47:42.224388 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30,"mutex_wait_us":49}
I20260807 08:47:42.224496 8038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:42.221157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.129:46375 (local address 127.4.159.130:47341)
0807 08:47:42.221338 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0807 08:47:42.221345 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0807 08:47:42.221397 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:47:42.221505 (+ 108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:47:42.221506 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:47:42.221508 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:47:42.221566 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:42.221567 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:42.222364 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:42.222364 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:42.223496 (+ 1132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:42.223499 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:42.224207 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:42.224208 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:42.224293 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:42.224297 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 08:47:42.224346 (+ 49us) client_negotiation.cc:240] Negotiation successful
0807 08:47:42.224382 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"mutex_wait_us":38,"thread_start_us":74,"threads_started":1}
I20260807 08:47:42.224843 7935 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f107b3c597cb4bdca57052aa5f9930bd" candidate_uuid: "56aa53ddfa8d405db14e40d13c45f344" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f212e32e8044e8c9eb244945ad1c908" is_pre_election: true
I20260807 08:47:42.224915 7531 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f107b3c597cb4bdca57052aa5f9930bd" candidate_uuid: "56aa53ddfa8d405db14e40d13c45f344" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd480a8c763249ba8217c035bf4fc386" is_pre_election: true
I20260807 08:47:42.225019 7531 raft_consensus.cc:2454] T f107b3c597cb4bdca57052aa5f9930bd P fd480a8c763249ba8217c035bf4fc386 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56aa53ddfa8d405db14e40d13c45f344 in term 0.
I20260807 08:47:42.225021 7935 raft_consensus.cc:2454] T f107b3c597cb4bdca57052aa5f9930bd P 0f212e32e8044e8c9eb244945ad1c908 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56aa53ddfa8d405db14e40d13c45f344 in term 0.
I20260807 08:47:42.225234 7630 leader_election.cc:302] T f107b3c597cb4bdca57052aa5f9930bd P 56aa53ddfa8d405db14e40d13c45f344 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f212e32e8044e8c9eb244945ad1c908, 56aa53ddfa8d405db14e40d13c45f344; no voters:
I20260807 08:47:42.225343 8037 raft_consensus.cc:2781] T f107b3c597cb4bdca57052aa5f9930bd P 56aa53ddfa8d405db14e40d13c45f344 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:47:42.225399 8037 raft_consensus.cc:482] T f107b3c597cb4bdca57052aa5f9930bd P 56aa53ddfa8d405db14e40d13c45f344 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:47:42.225421 8037 raft_consensus.cc:3037] T f107b3c597cb4bdca57052aa5f9930bd P 56aa53ddfa8d405db14e40d13c45f344 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:47:42.226014 8037 raft_consensus.cc:504] T f107b3c597cb4bdca57052aa5f9930bd P 56aa53ddfa8d405db14e40d13c45f344 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f212e32e8044e8c9eb244945ad1c908" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 45951 } } peers { permanent_uuid: "fd480a8c763249ba8217c035bf4fc386" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 46375 } } peers { permanent_uuid: "56aa53ddfa8d405db14e40d13c45f344" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 35393 } }
I20260807 08:47:42.226126 8037 leader_election.cc:288] T f107b3c597cb4bdca57052aa5f9930bd P 56aa53ddfa8d405db14e40d13c45f344 [CANDIDATE]: Term 1 election: Requested vote from peers 0f212e32e8044e8c9eb244945ad1c908 (127.4.159.131:45951), fd480a8c763249ba8217c035bf4fc386 (127.4.159.129:46375)
I20260807 08:47:42.226274 7531 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f107b3c597cb4bdca57052aa5f9930bd" candidate_uuid: "56aa53ddfa8d405db14e40d13c45f344" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd480a8c763249ba8217c035bf4fc386"
I20260807 08:47:42.226302 7935 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f107b3c597cb4bdca57052aa5f9930bd" candidate_uuid: "56aa53ddfa8d405db14e40d13c45f344" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f212e32e8044e8c9eb244945ad1c908"
I20260807 08:47:42.226372 7935 raft_consensus.cc:3037] T f107b3c597cb4bdca57052aa5f9930bd P 0f212e32e8044e8c9eb244945ad1c908 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:47:42.226368 7531 raft_consensus.cc:3037] T f107b3c597cb4bdca57052aa5f9930bd P fd480a8c763249ba8217c035bf4fc386 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:47:42.227145 7935 raft_consensus.cc:2454] T f107b3c597cb4bdca57052aa5f9930bd P 0f212e32e8044e8c9eb244945ad1c908 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56aa53ddfa8d405db14e40d13c45f344 in term 1.
I20260807 08:47:42.227145 7531 raft_consensus.cc:2454] T f107b3c597cb4bdca57052aa5f9930bd P fd480a8c763249ba8217c035bf4fc386 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56aa53ddfa8d405db14e40d13c45f344 in term 1.
I20260807 08:47:42.227687 7633 leader_election.cc:302] T f107b3c597cb4bdca57052aa5f9930bd P 56aa53ddfa8d405db14e40d13c45f344 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56aa53ddfa8d405db14e40d13c45f344, fd480a8c763249ba8217c035bf4fc386; no voters:
I20260807 08:47:42.227783 8037 raft_consensus.cc:2781] T f107b3c597cb4bdca57052aa5f9930bd P 56aa53ddfa8d405db14e40d13c45f344 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:47:42.227908 8037 raft_consensus.cc:686] T f107b3c597cb4bdca57052aa5f9930bd P 56aa53ddfa8d405db14e40d13c45f344 [term 1 LEADER]: Becoming Leader. State: Replica: 56aa53ddfa8d405db14e40d13c45f344, State: Running, Role: LEADER
I20260807 08:47:42.227974 8037 consensus_queue.cc:237] T f107b3c597cb4bdca57052aa5f9930bd P 56aa53ddfa8d405db14e40d13c45f344 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f212e32e8044e8c9eb244945ad1c908" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 45951 } } peers { permanent_uuid: "fd480a8c763249ba8217c035bf4fc386" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 46375 } } peers { permanent_uuid: "56aa53ddfa8d405db14e40d13c45f344" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 35393 } }
I20260807 08:47:42.228695 7310 catalog_manager.cc:5495] T f107b3c597cb4bdca57052aa5f9930bd P 56aa53ddfa8d405db14e40d13c45f344 reported cstate change: term changed from 0 to 1, leader changed from <none> to 56aa53ddfa8d405db14e40d13c45f344 (127.4.159.130). New cstate: current_term: 1 leader_uuid: "56aa53ddfa8d405db14e40d13c45f344" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f212e32e8044e8c9eb244945ad1c908" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 45951 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fd480a8c763249ba8217c035bf4fc386" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 46375 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56aa53ddfa8d405db14e40d13c45f344" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 35393 } health_report { overall_health: HEALTHY } } }
W20260807 08:47:42.256148 7804 tablet.cc:2369] T f107b3c597cb4bdca57052aa5f9930bd P 56aa53ddfa8d405db14e40d13c45f344: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 07 08:47:42 dist-test-slave-6zbq krb5kdc[7240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092461, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.190@KRBTEST.COM for kudu/127.4.159.190@KRBTEST.COM
I20260807 08:47:42.265002 8046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:42.258967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:33213 (local address 127.0.0.1:51366)
0807 08:47:42.259148 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0807 08:47:42.259158 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 08:47:42.259224 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:47:42.259354 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:47:42.259355 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:47:42.259647 (+ 292us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:47:42.259694 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:42.259696 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:42.260303 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:42.260304 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:42.260815 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:42.260816 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:42.261131 (+ 315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:42.261133 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:42.261183 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:42.261623 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:47:42.261628 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:47:42.262906 (+ 1278us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:47:42.264059 (+ 1153us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:47:42.264066 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:47:42.264068 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:47:42.264248 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:47:42.264407 (+ 159us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:47:42.264408 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:47:42.264408 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:47:42.264459 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:47:42.264661 (+ 202us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:47:42.264669 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:47:42.264715 (+ 46us) client_negotiation.cc:769] Sending connection context
0807 08:47:42.264771 (+ 56us) client_negotiation.cc:240] Negotiation successful
0807 08:47:42.264888 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":90,"threads_started":1}
I20260807 08:47:42.265044 7380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:42.259025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51366 (local address 127.4.159.190:33213)
0807 08:47:42.259108 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:42.259111 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:42.259217 (+ 106us) server_negotiation.cc:407] Connection header received
0807 08:47:42.259279 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:42.259280 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:42.259290 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:42.259333 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:47:42.259768 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:42.260236 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:42.260887 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:42.261062 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:42.261126 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:42.262981 (+ 1855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:47:42.262982 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:47:42.262982 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:47:42.262989 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:47:42.263987 (+ 998us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:42.264308 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:42.264309 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:42.264309 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:42.264356 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:42.264505 (+ 149us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:42.264506 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:42.264506 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:42.264600 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:47:42.264663 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:42.264799 (+ 136us) server_negotiation.cc:299] Negotiation successful
0807 08:47:42.264946 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":82,"server-negotiator.queue_time_us":48}
I20260807 08:47:42.266803 7310 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.4.159.190@KRBTEST.COM'} at 127.0.0.1:51366:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 08:47:42.266978 7310 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:47:42.268409 7505 tablet_service.cc:1467] Processing CreateTablet for tablet f0fdaa964cbc4fc9afbb81115e3de392 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=56219afacc6b42f1aa167e94c7cd53b7]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 08:47:42.268489 7711 tablet_service.cc:1467] Processing CreateTablet for tablet f0fdaa964cbc4fc9afbb81115e3de392 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=56219afacc6b42f1aa167e94c7cd53b7]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 08:47:42.268481 7905 tablet_service.cc:1467] Processing CreateTablet for tablet f0fdaa964cbc4fc9afbb81115e3de392 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=56219afacc6b42f1aa167e94c7cd53b7]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 08:47:42.268595 7505 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f0fdaa964cbc4fc9afbb81115e3de392. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:47:42.268620 7711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f0fdaa964cbc4fc9afbb81115e3de392. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:47:42.268643 7905 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f0fdaa964cbc4fc9afbb81115e3de392. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:47:42.269846 8029 tablet_bootstrap.cc:492] T f0fdaa964cbc4fc9afbb81115e3de392 P 0f212e32e8044e8c9eb244945ad1c908: Bootstrap starting.
I20260807 08:47:42.270068 8030 tablet_bootstrap.cc:492] T f0fdaa964cbc4fc9afbb81115e3de392 P fd480a8c763249ba8217c035bf4fc386: Bootstrap starting.
I20260807 08:47:42.270073 8031 tablet_bootstrap.cc:492] T f0fdaa964cbc4fc9afbb81115e3de392 P 56aa53ddfa8d405db14e40d13c45f344: Bootstrap starting.
I20260807 08:47:42.270457 8029 tablet_bootstrap.cc:654] T f0fdaa964cbc4fc9afbb81115e3de392 P 0f212e32e8044e8c9eb244945ad1c908: Neither blocks nor log segments found. Creating new log.
I20260807 08:47:42.270501 8031 tablet_bootstrap.cc:654] T f0fdaa964cbc4fc9afbb81115e3de392 P 56aa53ddfa8d405db14e40d13c45f344: Neither blocks nor log segments found. Creating new log.
I20260807 08:47:42.270697 8030 tablet_bootstrap.cc:654] T f0fdaa964cbc4fc9afbb81115e3de392 P fd480a8c763249ba8217c035bf4fc386: Neither blocks nor log segments found. Creating new log.
I20260807 08:47:42.271176 8029 tablet_bootstrap.cc:492] T f0fdaa964cbc4fc9afbb81115e3de392 P 0f212e32e8044e8c9eb244945ad1c908: No bootstrap required, opened a new log
I20260807 08:47:42.271229 8029 ts_tablet_manager.cc:1397] T f0fdaa964cbc4fc9afbb81115e3de392 P 0f212e32e8044e8c9eb244945ad1c908: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260807 08:47:42.271330 8030 tablet_bootstrap.cc:492] T f0fdaa964cbc4fc9afbb81115e3de392 P fd480a8c763249ba8217c035bf4fc386: No bootstrap required, opened a new log
I20260807 08:47:42.271363 8030 ts_tablet_manager.cc:1397] T f0fdaa964cbc4fc9afbb81115e3de392 P fd480a8c763249ba8217c035bf4fc386: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 08:47:42.271373 8031 tablet_bootstrap.cc:492] T f0fdaa964cbc4fc9afbb81115e3de392 P 56aa53ddfa8d405db14e40d13c45f344: No bootstrap required, opened a new log
I20260807 08:47:42.271411 8031 ts_tablet_manager.cc:1397] T f0fdaa964cbc4fc9afbb81115e3de392 P 56aa53ddfa8d405db14e40d13c45f344: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260807 08:47:42.271373 8029 raft_consensus.cc:348] T f0fdaa964cbc4fc9afbb81115e3de392 P 0f212e32e8044e8c9eb244945ad1c908 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd480a8c763249ba8217c035bf4fc386" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 46375 } } peers { permanent_uuid: "56aa53ddfa8d405db14e40d13c45f344" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 35393 } } peers { permanent_uuid: "0f212e32e8044e8c9eb244945ad1c908" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 45951 } }
I20260807 08:47:42.271445 8029 raft_consensus.cc:374] T f0fdaa964cbc4fc9afbb81115e3de392 P 0f212e32e8044e8c9eb244945ad1c908 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:47:42.271463 8029 raft_consensus.cc:729] T f0fdaa964cbc4fc9afbb81115e3de392 P 0f212e32e8044e8c9eb244945ad1c908 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f212e32e8044e8c9eb244945ad1c908, State: Initialized, Role: FOLLOWER
I20260807 08:47:42.271472 8030 raft_consensus.cc:348] T f0fdaa964cbc4fc9afbb81115e3de392 P fd480a8c763249ba8217c035bf4fc386 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd480a8c763249ba8217c035bf4fc386" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 46375 } } peers { permanent_uuid: "56aa53ddfa8d405db14e40d13c45f344" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 35393 } } peers { permanent_uuid: "0f212e32e8044e8c9eb244945ad1c908" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 45951 } }
I20260807 08:47:42.271525 8030 raft_consensus.cc:374] T f0fdaa964cbc4fc9afbb81115e3de392 P fd480a8c763249ba8217c035bf4fc386 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:47:42.271514 8029 consensus_queue.cc:260] T f0fdaa964cbc4fc9afbb81115e3de392 P 0f212e32e8044e8c9eb244945ad1c908 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd480a8c763249ba8217c035bf4fc386" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 46375 } } peers { permanent_uuid: "56aa53ddfa8d405db14e40d13c45f344" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 35393 } } peers { permanent_uuid: "0f212e32e8044e8c9eb244945ad1c908" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 45951 } }
I20260807 08:47:42.271543 8030 raft_consensus.cc:729] T f0fdaa964cbc4fc9afbb81115e3de392 P fd480a8c763249ba8217c035bf4fc386 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd480a8c763249ba8217c035bf4fc386, State: Initialized, Role: FOLLOWER
I20260807 08:47:42.271541 8031 raft_consensus.cc:348] T f0fdaa964cbc4fc9afbb81115e3de392 P 56aa53ddfa8d405db14e40d13c45f344 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd480a8c763249ba8217c035bf4fc386" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 46375 } } peers { permanent_uuid: "56aa53ddfa8d405db14e40d13c45f344" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 35393 } } peers { permanent_uuid: "0f212e32e8044e8c9eb244945ad1c908" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 45951 } }
I20260807 08:47:42.271589 8030 consensus_queue.cc:260] T f0fdaa964cbc4fc9afbb81115e3de392 P fd480a8c763249ba8217c035bf4fc386 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd480a8c763249ba8217c035bf4fc386" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 46375 } } peers { permanent_uuid: "56aa53ddfa8d405db14e40d13c45f344" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 35393 } } peers { permanent_uuid: "0f212e32e8044e8c9eb244945ad1c908" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 45951 } }
I20260807 08:47:42.271603 8029 ts_tablet_manager.cc:1428] T f0fdaa964cbc4fc9afbb81115e3de392 P 0f212e32e8044e8c9eb244945ad1c908: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:42.271620 8031 raft_consensus.cc:374] T f0fdaa964cbc4fc9afbb81115e3de392 P 56aa53ddfa8d405db14e40d13c45f344 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:47:42.271639 8031 raft_consensus.cc:729] T f0fdaa964cbc4fc9afbb81115e3de392 P 56aa53ddfa8d405db14e40d13c45f344 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56aa53ddfa8d405db14e40d13c45f344, State: Initialized, Role: FOLLOWER
I20260807 08:47:42.271665 8030 ts_tablet_manager.cc:1428] T f0fdaa964cbc4fc9afbb81115e3de392 P fd480a8c763249ba8217c035bf4fc386: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:42.271682 8031 consensus_queue.cc:260] T f0fdaa964cbc4fc9afbb81115e3de392 P 56aa53ddfa8d405db14e40d13c45f344 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd480a8c763249ba8217c035bf4fc386" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 46375 } } peers { permanent_uuid: "56aa53ddfa8d405db14e40d13c45f344" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 35393 } } peers { permanent_uuid: "0f212e32e8044e8c9eb244945ad1c908" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 45951 } }
I20260807 08:47:42.271665 8035 tablet_replica.cc:439] T f0fdaa964cbc4fc9afbb81115e3de392 P 0f212e32e8044e8c9eb244945ad1c908: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd480a8c763249ba8217c035bf4fc386" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 46375 } } peers { permanent_uuid: "56aa53ddfa8d405db14e40d13c45f344" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 35393 } } peers { permanent_uuid: "0f212e32e8044e8c9eb244945ad1c908" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 45951 } } }
I20260807 08:47:42.271768 8031 ts_tablet_manager.cc:1428] T f0fdaa964cbc4fc9afbb81115e3de392 P 56aa53ddfa8d405db14e40d13c45f344: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260807 08:47:42.271763 8035 tablet_replica.cc:442] T f0fdaa964cbc4fc9afbb81115e3de392 P 0f212e32e8044e8c9eb244945ad1c908: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 08:47:42.271888 8037 tablet_replica.cc:439] T f0fdaa964cbc4fc9afbb81115e3de392 P 56aa53ddfa8d405db14e40d13c45f344: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd480a8c763249ba8217c035bf4fc386" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 46375 } } peers { permanent_uuid: "56aa53ddfa8d405db14e40d13c45f344" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 35393 } } peers { permanent_uuid: "0f212e32e8044e8c9eb244945ad1c908" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 45951 } } }
I20260807 08:47:42.271950 8037 tablet_replica.cc:442] T f0fdaa964cbc4fc9afbb81115e3de392 P 56aa53ddfa8d405db14e40d13c45f344: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 08:47:42.271917 8036 tablet_replica.cc:439] T f0fdaa964cbc4fc9afbb81115e3de392 P fd480a8c763249ba8217c035bf4fc386: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd480a8c763249ba8217c035bf4fc386" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 46375 } } peers { permanent_uuid: "56aa53ddfa8d405db14e40d13c45f344" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 35393 } } peers { permanent_uuid: "0f212e32e8044e8c9eb244945ad1c908" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 45951 } } }
I20260807 08:47:42.272001 8036 tablet_replica.cc:442] T f0fdaa964cbc4fc9afbb81115e3de392 P fd480a8c763249ba8217c035bf4fc386: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 08:47:42.310848 8035 raft_consensus.cc:482] T f0fdaa964cbc4fc9afbb81115e3de392 P 0f212e32e8044e8c9eb244945ad1c908 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:47:42.310959 8035 raft_consensus.cc:504] T f0fdaa964cbc4fc9afbb81115e3de392 P 0f212e32e8044e8c9eb244945ad1c908 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd480a8c763249ba8217c035bf4fc386" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 46375 } } peers { permanent_uuid: "56aa53ddfa8d405db14e40d13c45f344" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 35393 } } peers { permanent_uuid: "0f212e32e8044e8c9eb244945ad1c908" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 45951 } }
I20260807 08:47:42.311322 8035 leader_election.cc:288] T f0fdaa964cbc4fc9afbb81115e3de392 P 0f212e32e8044e8c9eb244945ad1c908 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fd480a8c763249ba8217c035bf4fc386 (127.4.159.129:46375), 56aa53ddfa8d405db14e40d13c45f344 (127.4.159.130:35393)
I20260807 08:47:42.313951 7849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:42.311485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.130:35393 (local address 127.4.159.131:53283)
0807 08:47:42.311561 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0807 08:47:42.311574 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:47:42.311621 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:47:42.311758 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:47:42.311759 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:47:42.311761 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:47:42.311829 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:42.311831 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:42.312488 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:42.312489 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:42.313205 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:42.313207 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:42.313713 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:42.313714 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:42.313790 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:42.313793 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 08:47:42.313836 (+ 43us) client_negotiation.cc:240] Negotiation successful
0807 08:47:42.313876 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260807 08:47:42.313949 8028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:42.311595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.131:53283 (local address 127.4.159.130:35393)
0807 08:47:42.311653 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:42.311655 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:42.311663 (+ 8us) server_negotiation.cc:407] Connection header received
0807 08:47:42.311689 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:42.311689 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:42.311698 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:42.311743 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:47:42.311888 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:42.312413 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:42.313285 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:42.313648 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:42.313707 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:42.313716 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:42.313849 (+ 133us) server_negotiation.cc:299] Negotiation successful
0807 08:47:42.313881 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260807 08:47:42.313951 8025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:42.311566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.131:37667 (local address 127.4.159.129:46375)
0807 08:47:42.311614 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:42.311616 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:42.311747 (+ 131us) server_negotiation.cc:407] Connection header received
0807 08:47:42.311805 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:42.311806 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:42.311815 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:42.311872 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:47:42.312007 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:42.312461 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:42.313307 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:42.313646 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:42.313706 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:42.313715 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:42.313847 (+ 132us) server_negotiation.cc:299] Negotiation successful
0807 08:47:42.313883 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260807 08:47:42.313974 8056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:42.311485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.129:46375 (local address 127.4.159.131:37667)
0807 08:47:42.311679 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0807 08:47:42.311687 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0807 08:47:42.311745 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:47:42.311879 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:47:42.311879 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:47:42.311881 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:47:42.311952 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:42.311953 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:42.312531 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:42.312532 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:42.313227 (+ 695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:42.313229 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:42.313734 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:42.313735 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:42.313789 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:42.313792 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 08:47:42.313835 (+ 43us) client_negotiation.cc:240] Negotiation successful
0807 08:47:42.313879 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"mutex_wait_us":17,"thread_start_us":119,"threads_started":1}
I20260807 08:47:42.314252 7746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0fdaa964cbc4fc9afbb81115e3de392" candidate_uuid: "0f212e32e8044e8c9eb244945ad1c908" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56aa53ddfa8d405db14e40d13c45f344" is_pre_election: true
I20260807 08:47:42.314415 7746 raft_consensus.cc:2454] T f0fdaa964cbc4fc9afbb81115e3de392 P 56aa53ddfa8d405db14e40d13c45f344 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f212e32e8044e8c9eb244945ad1c908 in term 0.
I20260807 08:47:42.314448 7531 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0fdaa964cbc4fc9afbb81115e3de392" candidate_uuid: "0f212e32e8044e8c9eb244945ad1c908" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd480a8c763249ba8217c035bf4fc386" is_pre_election: true
I20260807 08:47:42.314541 7531 raft_consensus.cc:2454] T f0fdaa964cbc4fc9afbb81115e3de392 P fd480a8c763249ba8217c035bf4fc386 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f212e32e8044e8c9eb244945ad1c908 in term 0.
I20260807 08:47:42.314633 7836 leader_election.cc:302] T f0fdaa964cbc4fc9afbb81115e3de392 P 0f212e32e8044e8c9eb244945ad1c908 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f212e32e8044e8c9eb244945ad1c908, 56aa53ddfa8d405db14e40d13c45f344; no voters:
I20260807 08:47:42.314749 8035 raft_consensus.cc:2781] T f0fdaa964cbc4fc9afbb81115e3de392 P 0f212e32e8044e8c9eb244945ad1c908 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:47:42.314792 8035 raft_consensus.cc:482] T f0fdaa964cbc4fc9afbb81115e3de392 P 0f212e32e8044e8c9eb244945ad1c908 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:47:42.314806 8035 raft_consensus.cc:3037] T f0fdaa964cbc4fc9afbb81115e3de392 P 0f212e32e8044e8c9eb244945ad1c908 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:47:42.315456 8035 raft_consensus.cc:504] T f0fdaa964cbc4fc9afbb81115e3de392 P 0f212e32e8044e8c9eb244945ad1c908 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd480a8c763249ba8217c035bf4fc386" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 46375 } } peers { permanent_uuid: "56aa53ddfa8d405db14e40d13c45f344" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 35393 } } peers { permanent_uuid: "0f212e32e8044e8c9eb244945ad1c908" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 45951 } }
I20260807 08:47:42.315563 8035 leader_election.cc:288] T f0fdaa964cbc4fc9afbb81115e3de392 P 0f212e32e8044e8c9eb244945ad1c908 [CANDIDATE]: Term 1 election: Requested vote from peers fd480a8c763249ba8217c035bf4fc386 (127.4.159.129:46375), 56aa53ddfa8d405db14e40d13c45f344 (127.4.159.130:35393)
I20260807 08:47:42.315712 7531 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0fdaa964cbc4fc9afbb81115e3de392" candidate_uuid: "0f212e32e8044e8c9eb244945ad1c908" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd480a8c763249ba8217c035bf4fc386"
I20260807 08:47:42.315773 7746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0fdaa964cbc4fc9afbb81115e3de392" candidate_uuid: "0f212e32e8044e8c9eb244945ad1c908" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56aa53ddfa8d405db14e40d13c45f344"
I20260807 08:47:42.315810 7531 raft_consensus.cc:3037] T f0fdaa964cbc4fc9afbb81115e3de392 P fd480a8c763249ba8217c035bf4fc386 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:47:42.315833 7746 raft_consensus.cc:3037] T f0fdaa964cbc4fc9afbb81115e3de392 P 56aa53ddfa8d405db14e40d13c45f344 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:47:42.316572 7746 raft_consensus.cc:2454] T f0fdaa964cbc4fc9afbb81115e3de392 P 56aa53ddfa8d405db14e40d13c45f344 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f212e32e8044e8c9eb244945ad1c908 in term 1.
I20260807 08:47:42.316572 7531 raft_consensus.cc:2454] T f0fdaa964cbc4fc9afbb81115e3de392 P fd480a8c763249ba8217c035bf4fc386 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f212e32e8044e8c9eb244945ad1c908 in term 1.
I20260807 08:47:42.316743 7838 leader_election.cc:302] T f0fdaa964cbc4fc9afbb81115e3de392 P 0f212e32e8044e8c9eb244945ad1c908 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f212e32e8044e8c9eb244945ad1c908, fd480a8c763249ba8217c035bf4fc386; no voters:
I20260807 08:47:42.316838 8035 raft_consensus.cc:2781] T f0fdaa964cbc4fc9afbb81115e3de392 P 0f212e32e8044e8c9eb244945ad1c908 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:47:42.317003 8035 raft_consensus.cc:686] T f0fdaa964cbc4fc9afbb81115e3de392 P 0f212e32e8044e8c9eb244945ad1c908 [term 1 LEADER]: Becoming Leader. State: Replica: 0f212e32e8044e8c9eb244945ad1c908, State: Running, Role: LEADER
I20260807 08:47:42.317091 8035 consensus_queue.cc:237] T f0fdaa964cbc4fc9afbb81115e3de392 P 0f212e32e8044e8c9eb244945ad1c908 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd480a8c763249ba8217c035bf4fc386" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 46375 } } peers { permanent_uuid: "56aa53ddfa8d405db14e40d13c45f344" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 35393 } } peers { permanent_uuid: "0f212e32e8044e8c9eb244945ad1c908" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 45951 } }
I20260807 08:47:42.317553 8058 tablet_replica.cc:439] T f0fdaa964cbc4fc9afbb81115e3de392 P 0f212e32e8044e8c9eb244945ad1c908: TxnStatusTablet state changed. Reason: New leader 0f212e32e8044e8c9eb244945ad1c908. Latest consensus state: current_term: 1 leader_uuid: "0f212e32e8044e8c9eb244945ad1c908" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd480a8c763249ba8217c035bf4fc386" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 46375 } } peers { permanent_uuid: "56aa53ddfa8d405db14e40d13c45f344" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 35393 } } peers { permanent_uuid: "0f212e32e8044e8c9eb244945ad1c908" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 45951 } } }
I20260807 08:47:42.317623 8058 tablet_replica.cc:442] T f0fdaa964cbc4fc9afbb81115e3de392 P 0f212e32e8044e8c9eb244945ad1c908: This TxnStatusTablet replica's current role is: LEADER
I20260807 08:47:42.317797 8060 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 08:47:42.317898 8060 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 08:47:42.317868 7311 catalog_manager.cc:5495] T f0fdaa964cbc4fc9afbb81115e3de392 P 0f212e32e8044e8c9eb244945ad1c908 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0f212e32e8044e8c9eb244945ad1c908 (127.4.159.131). New cstate: current_term: 1 leader_uuid: "0f212e32e8044e8c9eb244945ad1c908" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd480a8c763249ba8217c035bf4fc386" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 46375 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56aa53ddfa8d405db14e40d13c45f344" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 35393 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f212e32e8044e8c9eb244945ad1c908" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 45951 } health_report { overall_health: HEALTHY } } }
I20260807 08:47:42.332994 8046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:42.330250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.131:45951 (local address 127.0.0.1:42098)
0807 08:47:42.330323 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0807 08:47:42.330336 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:47:42.330408 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:47:42.330565 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:47:42.330566 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:47:42.330567 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 08:47:42.330612 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:42.330615 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:42.331519 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:42.331520 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:42.332076 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:42.332078 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:42.332516 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:42.332517 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:42.332561 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:42.332577 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 08:47:42.332844 (+ 267us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 08:47:42.332854 (+ 10us) client_negotiation.cc:769] Sending connection context
0807 08:47:42.332894 (+ 40us) client_negotiation.cc:240] Negotiation successful
0807 08:47:42.332935 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260807 08:47:42.333016 8022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:42.330316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42098 (local address 127.4.159.131:45951)
0807 08:47:42.330391 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:42.330393 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:42.330403 (+ 10us) server_negotiation.cc:407] Connection header received
0807 08:47:42.330473 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:42.330474 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:42.330494 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:42.330564 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:47:42.330665 (+ 101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:42.331438 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:42.332155 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:42.332455 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:42.332521 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:42.332650 (+ 129us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:47:42.332773 (+ 123us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:47:42.332856 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:42.332898 (+ 42us) server_negotiation.cc:299] Negotiation successful
0807 08:47:42.332921 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 08:47:42.334790 7531 raft_consensus.cc:1261] T f0fdaa964cbc4fc9afbb81115e3de392 P fd480a8c763249ba8217c035bf4fc386 [term 1 FOLLOWER]: Refusing update from remote peer 0f212e32e8044e8c9eb244945ad1c908: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:47:42.334784 7746 raft_consensus.cc:1261] T f0fdaa964cbc4fc9afbb81115e3de392 P 56aa53ddfa8d405db14e40d13c45f344 [term 1 FOLLOWER]: Refusing update from remote peer 0f212e32e8044e8c9eb244945ad1c908: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:47:42.335100 8058 consensus_queue.cc:1035] T f0fdaa964cbc4fc9afbb81115e3de392 P 0f212e32e8044e8c9eb244945ad1c908 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56aa53ddfa8d405db14e40d13c45f344" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 35393 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:47:42.335273 8035 consensus_queue.cc:1035] T f0fdaa964cbc4fc9afbb81115e3de392 P 0f212e32e8044e8c9eb244945ad1c908 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd480a8c763249ba8217c035bf4fc386" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 46375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:47:42.336300 8037 tablet_replica.cc:439] T f0fdaa964cbc4fc9afbb81115e3de392 P 56aa53ddfa8d405db14e40d13c45f344: TxnStatusTablet state changed. Reason: New leader 0f212e32e8044e8c9eb244945ad1c908. Latest consensus state: current_term: 1 leader_uuid: "0f212e32e8044e8c9eb244945ad1c908" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd480a8c763249ba8217c035bf4fc386" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 46375 } } peers { permanent_uuid: "56aa53ddfa8d405db14e40d13c45f344" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 35393 } } peers { permanent_uuid: "0f212e32e8044e8c9eb244945ad1c908" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 45951 } } }
I20260807 08:47:42.336388 8037 tablet_replica.cc:442] T f0fdaa964cbc4fc9afbb81115e3de392 P 56aa53ddfa8d405db14e40d13c45f344: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 08:47:42.336804 8036 tablet_replica.cc:439] T f0fdaa964cbc4fc9afbb81115e3de392 P fd480a8c763249ba8217c035bf4fc386: TxnStatusTablet state changed. Reason: New leader 0f212e32e8044e8c9eb244945ad1c908. Latest consensus state: current_term: 1 leader_uuid: "0f212e32e8044e8c9eb244945ad1c908" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd480a8c763249ba8217c035bf4fc386" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 46375 } } peers { permanent_uuid: "56aa53ddfa8d405db14e40d13c45f344" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 35393 } } peers { permanent_uuid: "0f212e32e8044e8c9eb244945ad1c908" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 45951 } } }
I20260807 08:47:42.336896 8036 tablet_replica.cc:442] T f0fdaa964cbc4fc9afbb81115e3de392 P fd480a8c763249ba8217c035bf4fc386: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 08:47:42.337741 8058 tablet_replica.cc:439] T f0fdaa964cbc4fc9afbb81115e3de392 P 0f212e32e8044e8c9eb244945ad1c908: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0f212e32e8044e8c9eb244945ad1c908" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd480a8c763249ba8217c035bf4fc386" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 46375 } } peers { permanent_uuid: "56aa53ddfa8d405db14e40d13c45f344" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 35393 } } peers { permanent_uuid: "0f212e32e8044e8c9eb244945ad1c908" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 45951 } } }
I20260807 08:47:42.337836 8058 tablet_replica.cc:442] T f0fdaa964cbc4fc9afbb81115e3de392 P 0f212e32e8044e8c9eb244945ad1c908: This TxnStatusTablet replica's current role is: LEADER
I20260807 08:47:42.337858 8037 tablet_replica.cc:439] T f0fdaa964cbc4fc9afbb81115e3de392 P 56aa53ddfa8d405db14e40d13c45f344: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0f212e32e8044e8c9eb244945ad1c908" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd480a8c763249ba8217c035bf4fc386" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 46375 } } peers { permanent_uuid: "56aa53ddfa8d405db14e40d13c45f344" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 35393 } } peers { permanent_uuid: "0f212e32e8044e8c9eb244945ad1c908" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 45951 } } }
I20260807 08:47:42.337904 8064 mvcc.cc:205] Tried to move back new op lower bound from 7315834725720653824 to 7315834725651558400. Current Snapshot: MvccSnapshot[applied={T|T < 7315834725720653824}]
I20260807 08:47:42.337935 8037 tablet_replica.cc:442] T f0fdaa964cbc4fc9afbb81115e3de392 P 56aa53ddfa8d405db14e40d13c45f344: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 08:47:42.338127 8058 tablet_replica.cc:439] T f0fdaa964cbc4fc9afbb81115e3de392 P 0f212e32e8044e8c9eb244945ad1c908: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0f212e32e8044e8c9eb244945ad1c908" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd480a8c763249ba8217c035bf4fc386" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 46375 } } peers { permanent_uuid: "56aa53ddfa8d405db14e40d13c45f344" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 35393 } } peers { permanent_uuid: "0f212e32e8044e8c9eb244945ad1c908" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 45951 } } }
I20260807 08:47:42.338202 8058 tablet_replica.cc:442] T f0fdaa964cbc4fc9afbb81115e3de392 P 0f212e32e8044e8c9eb244945ad1c908: This TxnStatusTablet replica's current role is: LEADER
I20260807 08:47:42.338632 8066 mvcc.cc:205] Tried to move back new op lower bound from 7315834725720653824 to 7315834725651558400. Current Snapshot: MvccSnapshot[applied={T|T < 7315834725720653824}]
I20260807 08:47:42.338654 8036 tablet_replica.cc:439] T f0fdaa964cbc4fc9afbb81115e3de392 P fd480a8c763249ba8217c035bf4fc386: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0f212e32e8044e8c9eb244945ad1c908" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd480a8c763249ba8217c035bf4fc386" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 46375 } } peers { permanent_uuid: "56aa53ddfa8d405db14e40d13c45f344" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 35393 } } peers { permanent_uuid: "0f212e32e8044e8c9eb244945ad1c908" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 45951 } } }
I20260807 08:47:42.338737 8036 tablet_replica.cc:442] T f0fdaa964cbc4fc9afbb81115e3de392 P fd480a8c763249ba8217c035bf4fc386: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 08:47:42.339226 8068 mvcc.cc:205] Tried to move back new op lower bound from 7315834725720653824 to 7315834725651558400. Current Snapshot: MvccSnapshot[applied={T|T < 7315834725720653824}]
I20260807 08:47:42.343657 8028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:42.340991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55868 (local address 127.4.159.130:35393)
0807 08:47:42.341025 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:42.341027 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:42.341054 (+ 27us) server_negotiation.cc:407] Connection header received
0807 08:47:42.341145 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:42.341145 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:42.341160 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:42.341214 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:47:42.341325 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:42.341781 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:42.342912 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:42.343101 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:42.343149 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:42.343359 (+ 210us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:47:42.343460 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:47:42.343531 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:42.343585 (+ 54us) server_negotiation.cc:299] Negotiation successful
0807 08:47:42.343605 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260807 08:47:42.348225 8022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:42.346154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.130:40717 (local address 127.4.159.131:45951)
0807 08:47:42.346194 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:42.346196 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:42.346204 (+ 8us) server_negotiation.cc:407] Connection header received
0807 08:47:42.346253 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:42.346254 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:42.346264 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:42.346308 (+ 44us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:47:42.346462 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:42.346914 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:42.347613 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:42.347793 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:42.347854 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:42.348010 (+ 156us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:47:42.348042 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:47:42.348082 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:42.348132 (+ 50us) server_negotiation.cc:299] Negotiation successful
0807 08:47:42.348153 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 08:47:42.348232 7814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:42.346083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.131:45951 (local address 127.4.159.130:40717)
0807 08:47:42.346149 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0807 08:47:42.346160 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 08:47:42.346205 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:47:42.346309 (+ 104us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:47:42.346309 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:47:42.346310 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 08:47:42.346405 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:42.346407 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:42.346984 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:42.346985 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:42.347541 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:42.347543 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:42.347852 (+ 309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:42.347853 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:42.347916 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:42.347931 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 08:47:42.348089 (+ 158us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 08:47:42.348091 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 08:47:42.348130 (+ 39us) client_negotiation.cc:240] Negotiation successful
0807 08:47:42.348169 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260807 08:47:42.350528 7531 raft_consensus.cc:1261] T f107b3c597cb4bdca57052aa5f9930bd P fd480a8c763249ba8217c035bf4fc386 [term 1 FOLLOWER]: Refusing update from remote peer 56aa53ddfa8d405db14e40d13c45f344: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:47:42.350756 7935 raft_consensus.cc:1261] T f107b3c597cb4bdca57052aa5f9930bd P 0f212e32e8044e8c9eb244945ad1c908 [term 1 FOLLOWER]: Refusing update from remote peer 56aa53ddfa8d405db14e40d13c45f344: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:47:42.350800 8039 consensus_queue.cc:1035] T f107b3c597cb4bdca57052aa5f9930bd P 56aa53ddfa8d405db14e40d13c45f344 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd480a8c763249ba8217c035bf4fc386" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 46375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:47:42.350993 8039 consensus_queue.cc:1035] T f107b3c597cb4bdca57052aa5f9930bd P 56aa53ddfa8d405db14e40d13c45f344 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f212e32e8044e8c9eb244945ad1c908" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 45951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:47:42.365506 8028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:42.362915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.131:45325 (local address 127.4.159.130:35393)
0807 08:47:42.362954 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:42.362956 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:42.362962 (+ 6us) server_negotiation.cc:407] Connection header received
0807 08:47:42.362981 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:42.362982 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:42.362993 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:42.363041 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:47:42.363237 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:42.363741 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:42.364882 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:42.365117 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:42.365163 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:42.365328 (+ 165us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:47:42.365352 (+ 24us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:47:42.365383 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:42.365436 (+ 53us) server_negotiation.cc:299] Negotiation successful
0807 08:47:42.365452 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 08:47:42.365550 8006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:42.362710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.130:35393 (local address 127.4.159.131:45325)
0807 08:47:42.362780 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0807 08:47:42.362798 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 08:47:42.362842 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:47:42.363061 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:47:42.363062 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:47:42.363064 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 08:47:42.363146 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:42.363147 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:42.363820 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:42.363821 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:42.364800 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:42.364804 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:42.365169 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:42.365170 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:42.365240 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:42.365256 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 08:47:42.365386 (+ 130us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 08:47:42.365388 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 08:47:42.365433 (+ 45us) client_negotiation.cc:240] Negotiation successful
0807 08:47:42.365475 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260807 08:47:42.376747 7310 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51364:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:47:42.376859 7310 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51364:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:47:42.377565 7310 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d9cac9fdcacc45b693393f03561f7b11: Sending DeleteTablet for 3 replicas of tablet f107b3c597cb4bdca57052aa5f9930bd
I20260807 08:47:42.377964 7505 tablet_service.cc:1514] Processing DeleteTablet for tablet f107b3c597cb4bdca57052aa5f9930bd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:47:42 UTC) from {username='kudu', principal='kudu/127.4.159.190@KRBTEST.COM'} at 127.0.0.1:57780
I20260807 08:47:42.378129 7711 tablet_service.cc:1514] Processing DeleteTablet for tablet f107b3c597cb4bdca57052aa5f9930bd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:47:42 UTC) from {username='kudu', principal='kudu/127.4.159.190@KRBTEST.COM'} at 127.0.0.1:55864
I20260807 08:47:42.378362 8091 tablet_replica.cc:330] T f107b3c597cb4bdca57052aa5f9930bd P 56aa53ddfa8d405db14e40d13c45f344: stopping tablet replica
I20260807 08:47:42.378381 7905 tablet_service.cc:1514] Processing DeleteTablet for tablet f107b3c597cb4bdca57052aa5f9930bd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:47:42 UTC) from {username='kudu', principal='kudu/127.4.159.190@KRBTEST.COM'} at 127.0.0.1:42090
I20260807 08:47:42.378548 8091 raft_consensus.cc:2229] T f107b3c597cb4bdca57052aa5f9930bd P 56aa53ddfa8d405db14e40d13c45f344 [term 1 LEADER]: Raft consensus shutting down.
I20260807 08:47:42.378685 8091 raft_consensus.cc:2258] T f107b3c597cb4bdca57052aa5f9930bd P 56aa53ddfa8d405db14e40d13c45f344 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:47:42.378829 7310 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51364:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 08:47:42.378968 7310 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:47:42.379465 8091 ts_tablet_manager.cc:1905] T f107b3c597cb4bdca57052aa5f9930bd P 56aa53ddfa8d405db14e40d13c45f344: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:47:42.380091 7505 tablet_service.cc:1467] Processing CreateTablet for tablet 136c850b25584d3aaabb726f4f94c15f (DEFAULT_TABLE table=test-table [id=0d5567362b6946bd87e1da57077ca5d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:47:42.380244 7505 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 136c850b25584d3aaabb726f4f94c15f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:47:42.381443 8091 ts_tablet_manager.cc:1918] T f107b3c597cb4bdca57052aa5f9930bd P 56aa53ddfa8d405db14e40d13c45f344: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 08:47:42.381588 8091 log.cc:1196] T f107b3c597cb4bdca57052aa5f9930bd P 56aa53ddfa8d405db14e40d13c45f344: Deleting WAL directory at /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786092453486433-4734-0/minicluster-data/ts-1/wal/wals/f107b3c597cb4bdca57052aa5f9930bd
I20260807 08:47:42.381593 8030 tablet_bootstrap.cc:492] T 136c850b25584d3aaabb726f4f94c15f P fd480a8c763249ba8217c035bf4fc386: Bootstrap starting.
I20260807 08:47:42.381919 8091 ts_tablet_manager.cc:1939] T f107b3c597cb4bdca57052aa5f9930bd P 56aa53ddfa8d405db14e40d13c45f344: Deleting consensus metadata
I20260807 08:47:42.381973 8030 tablet_bootstrap.cc:654] T 136c850b25584d3aaabb726f4f94c15f P fd480a8c763249ba8217c035bf4fc386: Neither blocks nor log segments found. Creating new log.
I20260807 08:47:42.382483 7289 catalog_manager.cc:4841] TS 56aa53ddfa8d405db14e40d13c45f344 (127.4.159.130:35393): tablet f107b3c597cb4bdca57052aa5f9930bd (table test-table [id=d114a1b1fd124ae596ed4ab61ab93df9]) successfully deleted
I20260807 08:47:42.382625 7711 tablet_service.cc:1467] Processing CreateTablet for tablet 136c850b25584d3aaabb726f4f94c15f (DEFAULT_TABLE table=test-table [id=0d5567362b6946bd87e1da57077ca5d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:47:42.382725 7711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 136c850b25584d3aaabb726f4f94c15f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:47:42.382776 8030 tablet_bootstrap.cc:492] T 136c850b25584d3aaabb726f4f94c15f P fd480a8c763249ba8217c035bf4fc386: No bootstrap required, opened a new log
I20260807 08:47:42.382822 8030 ts_tablet_manager.cc:1397] T 136c850b25584d3aaabb726f4f94c15f P fd480a8c763249ba8217c035bf4fc386: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260807 08:47:42.382954 8030 raft_consensus.cc:348] T 136c850b25584d3aaabb726f4f94c15f P fd480a8c763249ba8217c035bf4fc386 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd480a8c763249ba8217c035bf4fc386" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 46375 } } peers { permanent_uuid: "56aa53ddfa8d405db14e40d13c45f344" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 35393 } } peers { permanent_uuid: "0f212e32e8044e8c9eb244945ad1c908" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 45951 } }
I20260807 08:47:42.383015 8030 raft_consensus.cc:374] T 136c850b25584d3aaabb726f4f94c15f P fd480a8c763249ba8217c035bf4fc386 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:47:42.383034 8030 raft_consensus.cc:729] T 136c850b25584d3aaabb726f4f94c15f P fd480a8c763249ba8217c035bf4fc386 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd480a8c763249ba8217c035bf4fc386, State: Initialized, Role: FOLLOWER
I20260807 08:47:42.383075 8030 consensus_queue.cc:260] T 136c850b25584d3aaabb726f4f94c15f P fd480a8c763249ba8217c035bf4fc386 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd480a8c763249ba8217c035bf4fc386" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 46375 } } peers { permanent_uuid: "56aa53ddfa8d405db14e40d13c45f344" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 35393 } } peers { permanent_uuid: "0f212e32e8044e8c9eb244945ad1c908" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 45951 } }
I20260807 08:47:42.383158 8030 ts_tablet_manager.cc:1428] T 136c850b25584d3aaabb726f4f94c15f P fd480a8c763249ba8217c035bf4fc386: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:42.383863 8031 tablet_bootstrap.cc:492] T 136c850b25584d3aaabb726f4f94c15f P 56aa53ddfa8d405db14e40d13c45f344: Bootstrap starting.
I20260807 08:47:42.384083 7905 tablet_service.cc:1467] Processing CreateTablet for tablet 136c850b25584d3aaabb726f4f94c15f (DEFAULT_TABLE table=test-table [id=0d5567362b6946bd87e1da57077ca5d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:47:42.384186 7905 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 136c850b25584d3aaabb726f4f94c15f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:47:42.384268 8031 tablet_bootstrap.cc:654] T 136c850b25584d3aaabb726f4f94c15f P 56aa53ddfa8d405db14e40d13c45f344: Neither blocks nor log segments found. Creating new log.
I20260807 08:47:42.385352 8029 tablet_bootstrap.cc:492] T 136c850b25584d3aaabb726f4f94c15f P 0f212e32e8044e8c9eb244945ad1c908: Bootstrap starting.
I20260807 08:47:42.385619 8029 tablet_bootstrap.cc:654] T 136c850b25584d3aaabb726f4f94c15f P 0f212e32e8044e8c9eb244945ad1c908: Neither blocks nor log segments found. Creating new log.
I20260807 08:47:42.386163 8029 tablet_bootstrap.cc:492] T 136c850b25584d3aaabb726f4f94c15f P 0f212e32e8044e8c9eb244945ad1c908: No bootstrap required, opened a new log
I20260807 08:47:42.386224 8029 ts_tablet_manager.cc:1397] T 136c850b25584d3aaabb726f4f94c15f P 0f212e32e8044e8c9eb244945ad1c908: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260807 08:47:42.386319 8029 raft_consensus.cc:348] T 136c850b25584d3aaabb726f4f94c15f P 0f212e32e8044e8c9eb244945ad1c908 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd480a8c763249ba8217c035bf4fc386" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 46375 } } peers { permanent_uuid: "56aa53ddfa8d405db14e40d13c45f344" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 35393 } } peers { permanent_uuid: "0f212e32e8044e8c9eb244945ad1c908" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 45951 } }
I20260807 08:47:42.386400 8029 raft_consensus.cc:374] T 136c850b25584d3aaabb726f4f94c15f P 0f212e32e8044e8c9eb244945ad1c908 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:47:42.386421 8029 raft_consensus.cc:729] T 136c850b25584d3aaabb726f4f94c15f P 0f212e32e8044e8c9eb244945ad1c908 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f212e32e8044e8c9eb244945ad1c908, State: Initialized, Role: FOLLOWER
I20260807 08:47:42.386459 8029 consensus_queue.cc:260] T 136c850b25584d3aaabb726f4f94c15f P 0f212e32e8044e8c9eb244945ad1c908 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd480a8c763249ba8217c035bf4fc386" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 46375 } } peers { permanent_uuid: "56aa53ddfa8d405db14e40d13c45f344" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 35393 } } peers { permanent_uuid: "0f212e32e8044e8c9eb244945ad1c908" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 45951 } }
I20260807 08:47:42.386535 8029 ts_tablet_manager.cc:1428] T 136c850b25584d3aaabb726f4f94c15f P 0f212e32e8044e8c9eb244945ad1c908: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:42.386642 8090 tablet_replica.cc:330] T f107b3c597cb4bdca57052aa5f9930bd P fd480a8c763249ba8217c035bf4fc386: stopping tablet replica
I20260807 08:47:42.387086 8090 raft_consensus.cc:2229] T f107b3c597cb4bdca57052aa5f9930bd P fd480a8c763249ba8217c035bf4fc386 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:47:42.387164 8090 raft_consensus.cc:2258] T f107b3c597cb4bdca57052aa5f9930bd P fd480a8c763249ba8217c035bf4fc386 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:47:42.387396 8092 tablet_replica.cc:330] T f107b3c597cb4bdca57052aa5f9930bd P 0f212e32e8044e8c9eb244945ad1c908: stopping tablet replica
I20260807 08:47:42.387490 8092 raft_consensus.cc:2229] T f107b3c597cb4bdca57052aa5f9930bd P 0f212e32e8044e8c9eb244945ad1c908 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:47:42.387547 8092 raft_consensus.cc:2258] T f107b3c597cb4bdca57052aa5f9930bd P 0f212e32e8044e8c9eb244945ad1c908 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:47:42.387843 8090 ts_tablet_manager.cc:1905] T f107b3c597cb4bdca57052aa5f9930bd P fd480a8c763249ba8217c035bf4fc386: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:47:42.389708 8090 ts_tablet_manager.cc:1918] T f107b3c597cb4bdca57052aa5f9930bd P fd480a8c763249ba8217c035bf4fc386: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 08:47:42.390398 8090 log.cc:1196] T f107b3c597cb4bdca57052aa5f9930bd P fd480a8c763249ba8217c035bf4fc386: Deleting WAL directory at /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786092453486433-4734-0/minicluster-data/ts-0/wal/wals/f107b3c597cb4bdca57052aa5f9930bd
I20260807 08:47:42.390456 8092 ts_tablet_manager.cc:1905] T f107b3c597cb4bdca57052aa5f9930bd P 0f212e32e8044e8c9eb244945ad1c908: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:47:42.390770 8090 ts_tablet_manager.cc:1939] T f107b3c597cb4bdca57052aa5f9930bd P fd480a8c763249ba8217c035bf4fc386: Deleting consensus metadata
I20260807 08:47:42.391394 7292 catalog_manager.cc:4841] TS fd480a8c763249ba8217c035bf4fc386 (127.4.159.129:46375): tablet f107b3c597cb4bdca57052aa5f9930bd (table test-table [id=d114a1b1fd124ae596ed4ab61ab93df9]) successfully deleted
I20260807 08:47:42.391686 8031 tablet_bootstrap.cc:492] T 136c850b25584d3aaabb726f4f94c15f P 56aa53ddfa8d405db14e40d13c45f344: No bootstrap required, opened a new log
I20260807 08:47:42.391732 8031 ts_tablet_manager.cc:1397] T 136c850b25584d3aaabb726f4f94c15f P 56aa53ddfa8d405db14e40d13c45f344: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.001s
I20260807 08:47:42.391860 8092 ts_tablet_manager.cc:1918] T f107b3c597cb4bdca57052aa5f9930bd P 0f212e32e8044e8c9eb244945ad1c908: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 08:47:42.391866 8031 raft_consensus.cc:348] T 136c850b25584d3aaabb726f4f94c15f P 56aa53ddfa8d405db14e40d13c45f344 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd480a8c763249ba8217c035bf4fc386" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 46375 } } peers { permanent_uuid: "56aa53ddfa8d405db14e40d13c45f344" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 35393 } } peers { permanent_uuid: "0f212e32e8044e8c9eb244945ad1c908" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 45951 } }
I20260807 08:47:42.391925 8031 raft_consensus.cc:374] T 136c850b25584d3aaabb726f4f94c15f P 56aa53ddfa8d405db14e40d13c45f344 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:47:42.391932 8092 log.cc:1196] T f107b3c597cb4bdca57052aa5f9930bd P 0f212e32e8044e8c9eb244945ad1c908: Deleting WAL directory at /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786092453486433-4734-0/minicluster-data/ts-2/wal/wals/f107b3c597cb4bdca57052aa5f9930bd
I20260807 08:47:42.391943 8031 raft_consensus.cc:729] T 136c850b25584d3aaabb726f4f94c15f P 56aa53ddfa8d405db14e40d13c45f344 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56aa53ddfa8d405db14e40d13c45f344, State: Initialized, Role: FOLLOWER
I20260807 08:47:42.391980 8031 consensus_queue.cc:260] T 136c850b25584d3aaabb726f4f94c15f P 56aa53ddfa8d405db14e40d13c45f344 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd480a8c763249ba8217c035bf4fc386" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 46375 } } peers { permanent_uuid: "56aa53ddfa8d405db14e40d13c45f344" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 35393 } } peers { permanent_uuid: "0f212e32e8044e8c9eb244945ad1c908" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 45951 } }
I20260807 08:47:42.392051 8031 ts_tablet_manager.cc:1428] T 136c850b25584d3aaabb726f4f94c15f P 56aa53ddfa8d405db14e40d13c45f344: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:42.392760 8092 ts_tablet_manager.cc:1939] T f107b3c597cb4bdca57052aa5f9930bd P 0f212e32e8044e8c9eb244945ad1c908: Deleting consensus metadata
I20260807 08:47:42.393122 7289 catalog_manager.cc:4841] TS 0f212e32e8044e8c9eb244945ad1c908 (127.4.159.131:45951): tablet f107b3c597cb4bdca57052aa5f9930bd (table test-table [id=d114a1b1fd124ae596ed4ab61ab93df9]) successfully deleted
W20260807 08:47:42.404367 7999 tablet.cc:2369] T 136c850b25584d3aaabb726f4f94c15f P 0f212e32e8044e8c9eb244945ad1c908: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 08:47:42.497521 8080 raft_consensus.cc:482] T 136c850b25584d3aaabb726f4f94c15f P 56aa53ddfa8d405db14e40d13c45f344 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:47:42.497623 8080 raft_consensus.cc:504] T 136c850b25584d3aaabb726f4f94c15f P 56aa53ddfa8d405db14e40d13c45f344 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd480a8c763249ba8217c035bf4fc386" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 46375 } } peers { permanent_uuid: "56aa53ddfa8d405db14e40d13c45f344" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 35393 } } peers { permanent_uuid: "0f212e32e8044e8c9eb244945ad1c908" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 45951 } }
I20260807 08:47:42.497761 8080 leader_election.cc:288] T 136c850b25584d3aaabb726f4f94c15f P 56aa53ddfa8d405db14e40d13c45f344 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fd480a8c763249ba8217c035bf4fc386 (127.4.159.129:46375), 0f212e32e8044e8c9eb244945ad1c908 (127.4.159.131:45951)
I20260807 08:47:42.498003 7531 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "136c850b25584d3aaabb726f4f94c15f" candidate_uuid: "56aa53ddfa8d405db14e40d13c45f344" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd480a8c763249ba8217c035bf4fc386" is_pre_election: true
I20260807 08:47:42.498003 7935 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "136c850b25584d3aaabb726f4f94c15f" candidate_uuid: "56aa53ddfa8d405db14e40d13c45f344" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f212e32e8044e8c9eb244945ad1c908" is_pre_election: true
I20260807 08:47:42.498121 7935 raft_consensus.cc:2454] T 136c850b25584d3aaabb726f4f94c15f P 0f212e32e8044e8c9eb244945ad1c908 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56aa53ddfa8d405db14e40d13c45f344 in term 0.
I20260807 08:47:42.498121 7531 raft_consensus.cc:2454] T 136c850b25584d3aaabb726f4f94c15f P fd480a8c763249ba8217c035bf4fc386 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56aa53ddfa8d405db14e40d13c45f344 in term 0.
I20260807 08:47:42.498281 7630 leader_election.cc:302] T 136c850b25584d3aaabb726f4f94c15f P 56aa53ddfa8d405db14e40d13c45f344 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f212e32e8044e8c9eb244945ad1c908, 56aa53ddfa8d405db14e40d13c45f344; no voters:
I20260807 08:47:42.498409 8080 raft_consensus.cc:2781] T 136c850b25584d3aaabb726f4f94c15f P 56aa53ddfa8d405db14e40d13c45f344 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:47:42.498449 8080 raft_consensus.cc:482] T 136c850b25584d3aaabb726f4f94c15f P 56aa53ddfa8d405db14e40d13c45f344 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:47:42.498462 8080 raft_consensus.cc:3037] T 136c850b25584d3aaabb726f4f94c15f P 56aa53ddfa8d405db14e40d13c45f344 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:47:42.499075 8080 raft_consensus.cc:504] T 136c850b25584d3aaabb726f4f94c15f P 56aa53ddfa8d405db14e40d13c45f344 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd480a8c763249ba8217c035bf4fc386" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 46375 } } peers { permanent_uuid: "56aa53ddfa8d405db14e40d13c45f344" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 35393 } } peers { permanent_uuid: "0f212e32e8044e8c9eb244945ad1c908" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 45951 } }
I20260807 08:47:42.499209 8080 leader_election.cc:288] T 136c850b25584d3aaabb726f4f94c15f P 56aa53ddfa8d405db14e40d13c45f344 [CANDIDATE]: Term 1 election: Requested vote from peers fd480a8c763249ba8217c035bf4fc386 (127.4.159.129:46375), 0f212e32e8044e8c9eb244945ad1c908 (127.4.159.131:45951)
I20260807 08:47:42.499353 7531 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "136c850b25584d3aaabb726f4f94c15f" candidate_uuid: "56aa53ddfa8d405db14e40d13c45f344" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd480a8c763249ba8217c035bf4fc386"
I20260807 08:47:42.499433 7531 raft_consensus.cc:3037] T 136c850b25584d3aaabb726f4f94c15f P fd480a8c763249ba8217c035bf4fc386 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:47:42.499413 7935 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "136c850b25584d3aaabb726f4f94c15f" candidate_uuid: "56aa53ddfa8d405db14e40d13c45f344" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f212e32e8044e8c9eb244945ad1c908"
I20260807 08:47:42.499495 7935 raft_consensus.cc:3037] T 136c850b25584d3aaabb726f4f94c15f P 0f212e32e8044e8c9eb244945ad1c908 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:47:42.500226 7531 raft_consensus.cc:2454] T 136c850b25584d3aaabb726f4f94c15f P fd480a8c763249ba8217c035bf4fc386 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56aa53ddfa8d405db14e40d13c45f344 in term 1.
I20260807 08:47:42.500226 7935 raft_consensus.cc:2454] T 136c850b25584d3aaabb726f4f94c15f P 0f212e32e8044e8c9eb244945ad1c908 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56aa53ddfa8d405db14e40d13c45f344 in term 1.
I20260807 08:47:42.500409 7633 leader_election.cc:302] T 136c850b25584d3aaabb726f4f94c15f P 56aa53ddfa8d405db14e40d13c45f344 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56aa53ddfa8d405db14e40d13c45f344, fd480a8c763249ba8217c035bf4fc386; no voters:
I20260807 08:47:42.500519 8080 raft_consensus.cc:2781] T 136c850b25584d3aaabb726f4f94c15f P 56aa53ddfa8d405db14e40d13c45f344 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:47:42.500586 8080 raft_consensus.cc:686] T 136c850b25584d3aaabb726f4f94c15f P 56aa53ddfa8d405db14e40d13c45f344 [term 1 LEADER]: Becoming Leader. State: Replica: 56aa53ddfa8d405db14e40d13c45f344, State: Running, Role: LEADER
I20260807 08:47:42.500636 8080 consensus_queue.cc:237] T 136c850b25584d3aaabb726f4f94c15f P 56aa53ddfa8d405db14e40d13c45f344 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd480a8c763249ba8217c035bf4fc386" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 46375 } } peers { permanent_uuid: "56aa53ddfa8d405db14e40d13c45f344" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 35393 } } peers { permanent_uuid: "0f212e32e8044e8c9eb244945ad1c908" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 45951 } }
I20260807 08:47:42.501186 7311 catalog_manager.cc:5495] T 136c850b25584d3aaabb726f4f94c15f P 56aa53ddfa8d405db14e40d13c45f344 reported cstate change: term changed from 0 to 1, leader changed from <none> to 56aa53ddfa8d405db14e40d13c45f344 (127.4.159.130). New cstate: current_term: 1 leader_uuid: "56aa53ddfa8d405db14e40d13c45f344" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd480a8c763249ba8217c035bf4fc386" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 46375 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56aa53ddfa8d405db14e40d13c45f344" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 35393 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f212e32e8044e8c9eb244945ad1c908" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 45951 } health_report { overall_health: UNKNOWN } } }
I20260807 08:47:42.535173 7935 raft_consensus.cc:1261] T 136c850b25584d3aaabb726f4f94c15f P 0f212e32e8044e8c9eb244945ad1c908 [term 1 FOLLOWER]: Refusing update from remote peer 56aa53ddfa8d405db14e40d13c45f344: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:47:42.535261 7531 raft_consensus.cc:1261] T 136c850b25584d3aaabb726f4f94c15f P fd480a8c763249ba8217c035bf4fc386 [term 1 FOLLOWER]: Refusing update from remote peer 56aa53ddfa8d405db14e40d13c45f344: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:47:42.535398 8080 consensus_queue.cc:1035] T 136c850b25584d3aaabb726f4f94c15f P 56aa53ddfa8d405db14e40d13c45f344 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f212e32e8044e8c9eb244945ad1c908" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 45951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:47:42.535503 8039 consensus_queue.cc:1035] T 136c850b25584d3aaabb726f4f94c15f P 56aa53ddfa8d405db14e40d13c45f344 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd480a8c763249ba8217c035bf4fc386" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 46375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 08:47:42.539822 8066 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260807 08:47:44.059231 7310 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51364:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:47:44.059324 7310 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51364:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:47:44.060056 7310 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d9cac9fdcacc45b693393f03561f7b11: Sending DeleteTablet for 3 replicas of tablet 136c850b25584d3aaabb726f4f94c15f
I20260807 08:47:44.060339 7505 tablet_service.cc:1514] Processing DeleteTablet for tablet 136c850b25584d3aaabb726f4f94c15f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:47:44 UTC) from {username='kudu', principal='kudu/127.4.159.190@KRBTEST.COM'} at 127.0.0.1:57780
I20260807 08:47:44.060374 7711 tablet_service.cc:1514] Processing DeleteTablet for tablet 136c850b25584d3aaabb726f4f94c15f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:47:44 UTC) from {username='kudu', principal='kudu/127.4.159.190@KRBTEST.COM'} at 127.0.0.1:55864
I20260807 08:47:44.060735 7904 tablet_service.cc:1514] Processing DeleteTablet for tablet 136c850b25584d3aaabb726f4f94c15f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:47:44 UTC) from {username='kudu', principal='kudu/127.4.159.190@KRBTEST.COM'} at 127.0.0.1:42090
I20260807 08:47:44.060904 8132 tablet_replica.cc:330] T 136c850b25584d3aaabb726f4f94c15f P 0f212e32e8044e8c9eb244945ad1c908: stopping tablet replica
I20260807 08:47:44.060914 8130 tablet_replica.cc:330] T 136c850b25584d3aaabb726f4f94c15f P fd480a8c763249ba8217c035bf4fc386: stopping tablet replica
I20260807 08:47:44.060964 8130 raft_consensus.cc:2229] T 136c850b25584d3aaabb726f4f94c15f P fd480a8c763249ba8217c035bf4fc386 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:47:44.060998 8132 raft_consensus.cc:2229] T 136c850b25584d3aaabb726f4f94c15f P 0f212e32e8044e8c9eb244945ad1c908 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:47:44.061064 8130 raft_consensus.cc:2258] T 136c850b25584d3aaabb726f4f94c15f P fd480a8c763249ba8217c035bf4fc386 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:47:44.061064 8132 raft_consensus.cc:2258] T 136c850b25584d3aaabb726f4f94c15f P 0f212e32e8044e8c9eb244945ad1c908 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:47:44.061225 8131 tablet_replica.cc:330] T 136c850b25584d3aaabb726f4f94c15f P 56aa53ddfa8d405db14e40d13c45f344: stopping tablet replica
I20260807 08:47:44.061406 8130 ts_tablet_manager.cc:1905] T 136c850b25584d3aaabb726f4f94c15f P fd480a8c763249ba8217c035bf4fc386: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:47:44.062530 8132 ts_tablet_manager.cc:1905] T 136c850b25584d3aaabb726f4f94c15f P 0f212e32e8044e8c9eb244945ad1c908: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:47:44.062911 8131 raft_consensus.cc:2229] T 136c850b25584d3aaabb726f4f94c15f P 56aa53ddfa8d405db14e40d13c45f344 [term 1 LEADER]: Raft consensus shutting down.
I20260807 08:47:44.063087 8130 ts_tablet_manager.cc:1918] T 136c850b25584d3aaabb726f4f94c15f P fd480a8c763249ba8217c035bf4fc386: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 08:47:44.063194 8130 log.cc:1196] T 136c850b25584d3aaabb726f4f94c15f P fd480a8c763249ba8217c035bf4fc386: Deleting WAL directory at /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786092453486433-4734-0/minicluster-data/ts-0/wal/wals/136c850b25584d3aaabb726f4f94c15f
I20260807 08:47:44.063441 8130 ts_tablet_manager.cc:1939] T 136c850b25584d3aaabb726f4f94c15f P fd480a8c763249ba8217c035bf4fc386: Deleting consensus metadata
I20260807 08:47:44.063738 7292 catalog_manager.cc:4841] TS fd480a8c763249ba8217c035bf4fc386 (127.4.159.129:46375): tablet 136c850b25584d3aaabb726f4f94c15f (table test-table [id=0d5567362b6946bd87e1da57077ca5d0]) successfully deleted
I20260807 08:47:44.063968 8132 ts_tablet_manager.cc:1918] T 136c850b25584d3aaabb726f4f94c15f P 0f212e32e8044e8c9eb244945ad1c908: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 08:47:44.064016 8132 log.cc:1196] T 136c850b25584d3aaabb726f4f94c15f P 0f212e32e8044e8c9eb244945ad1c908: Deleting WAL directory at /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786092453486433-4734-0/minicluster-data/ts-2/wal/wals/136c850b25584d3aaabb726f4f94c15f
I20260807 08:47:44.064198 8132 ts_tablet_manager.cc:1939] T 136c850b25584d3aaabb726f4f94c15f P 0f212e32e8044e8c9eb244945ad1c908: Deleting consensus metadata
I20260807 08:47:44.064474 7289 catalog_manager.cc:4841] TS 0f212e32e8044e8c9eb244945ad1c908 (127.4.159.131:45951): tablet 136c850b25584d3aaabb726f4f94c15f (table test-table [id=0d5567362b6946bd87e1da57077ca5d0]) successfully deleted
I20260807 08:47:44.065970 8131 raft_consensus.cc:2258] T 136c850b25584d3aaabb726f4f94c15f P 56aa53ddfa8d405db14e40d13c45f344 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:47:44.066196 8131 ts_tablet_manager.cc:1905] T 136c850b25584d3aaabb726f4f94c15f P 56aa53ddfa8d405db14e40d13c45f344: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:47:44.067481 8131 ts_tablet_manager.cc:1918] T 136c850b25584d3aaabb726f4f94c15f P 56aa53ddfa8d405db14e40d13c45f344: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 08:47:44.067543 8131 log.cc:1196] T 136c850b25584d3aaabb726f4f94c15f P 56aa53ddfa8d405db14e40d13c45f344: Deleting WAL directory at /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786092453486433-4734-0/minicluster-data/ts-1/wal/wals/136c850b25584d3aaabb726f4f94c15f
I20260807 08:47:44.067754 8131 ts_tablet_manager.cc:1939] T 136c850b25584d3aaabb726f4f94c15f P 56aa53ddfa8d405db14e40d13c45f344: Deleting consensus metadata
I20260807 08:47:44.068029 7289 catalog_manager.cc:4841] TS 56aa53ddfa8d405db14e40d13c45f344 (127.4.159.130:35393): tablet 136c850b25584d3aaabb726f4f94c15f (table test-table [id=0d5567362b6946bd87e1da57077ca5d0]) successfully deleted
Aug 07 08:47:44 dist-test-slave-6zbq krb5kdc[7240](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092464, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 08:47:44 dist-test-slave-6zbq krb5kdc[7240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092464, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.4.159.190@KRBTEST.COM
I20260807 08:47:44.087265 8142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:44.077021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51370 (local address 127.4.159.190:33213)
0807 08:47:44.077134 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:44.077136 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:44.077161 (+ 25us) server_negotiation.cc:407] Connection header received
0807 08:47:44.077213 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:44.077214 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:44.077223 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:44.077275 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:47:44.077766 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:44.078260 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:44.078954 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:44.079174 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:44.079244 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:44.085022 (+ 5778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:47:44.085023 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:47:44.085024 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:47:44.085030 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:47:44.086328 (+ 1298us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:44.086666 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:44.086667 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:44.086667 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:44.086718 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:44.086870 (+ 152us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:44.086870 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:44.086870 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:44.086960 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:47:44.087052 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:44.087138 (+ 86us) server_negotiation.cc:299] Negotiation successful
0807 08:47:44.087202 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90,"thread_start_us":69,"threads_started":1}
I20260807 08:47:44.088532 7310 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51370:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 08:47:44.088655 7310 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:47:44.089856 7505 tablet_service.cc:1467] Processing CreateTablet for tablet 0861cff26c754297aeef467f9d2bb05e (DEFAULT_TABLE table=test-table [id=2276d9b839164e41b5d4662b8c1f73c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:47:44.089867 7904 tablet_service.cc:1467] Processing CreateTablet for tablet 0861cff26c754297aeef467f9d2bb05e (DEFAULT_TABLE table=test-table [id=2276d9b839164e41b5d4662b8c1f73c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:47:44.089891 7711 tablet_service.cc:1467] Processing CreateTablet for tablet 0861cff26c754297aeef467f9d2bb05e (DEFAULT_TABLE table=test-table [id=2276d9b839164e41b5d4662b8c1f73c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:47:44.089977 7711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0861cff26c754297aeef467f9d2bb05e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:47:44.089975 7505 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0861cff26c754297aeef467f9d2bb05e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:47:44.089969 7904 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0861cff26c754297aeef467f9d2bb05e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:47:44.091300 8144 tablet_bootstrap.cc:492] T 0861cff26c754297aeef467f9d2bb05e P 0f212e32e8044e8c9eb244945ad1c908: Bootstrap starting.
I20260807 08:47:44.091321 8145 tablet_bootstrap.cc:492] T 0861cff26c754297aeef467f9d2bb05e P 56aa53ddfa8d405db14e40d13c45f344: Bootstrap starting.
I20260807 08:47:44.091614 8144 tablet_bootstrap.cc:654] T 0861cff26c754297aeef467f9d2bb05e P 0f212e32e8044e8c9eb244945ad1c908: Neither blocks nor log segments found. Creating new log.
I20260807 08:47:44.091626 8145 tablet_bootstrap.cc:654] T 0861cff26c754297aeef467f9d2bb05e P 56aa53ddfa8d405db14e40d13c45f344: Neither blocks nor log segments found. Creating new log.
I20260807 08:47:44.091878 8146 tablet_bootstrap.cc:492] T 0861cff26c754297aeef467f9d2bb05e P fd480a8c763249ba8217c035bf4fc386: Bootstrap starting.
I20260807 08:47:44.092130 8145 tablet_bootstrap.cc:492] T 0861cff26c754297aeef467f9d2bb05e P 56aa53ddfa8d405db14e40d13c45f344: No bootstrap required, opened a new log
I20260807 08:47:44.092175 8145 ts_tablet_manager.cc:1397] T 0861cff26c754297aeef467f9d2bb05e P 56aa53ddfa8d405db14e40d13c45f344: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 08:47:44.092211 8146 tablet_bootstrap.cc:654] T 0861cff26c754297aeef467f9d2bb05e P fd480a8c763249ba8217c035bf4fc386: Neither blocks nor log segments found. Creating new log.
I20260807 08:47:44.092291 8145 raft_consensus.cc:348] T 0861cff26c754297aeef467f9d2bb05e P 56aa53ddfa8d405db14e40d13c45f344 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd480a8c763249ba8217c035bf4fc386" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 46375 } } peers { permanent_uuid: "56aa53ddfa8d405db14e40d13c45f344" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 35393 } } peers { permanent_uuid: "0f212e32e8044e8c9eb244945ad1c908" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 45951 } }
I20260807 08:47:44.092355 8145 raft_consensus.cc:374] T 0861cff26c754297aeef467f9d2bb05e P 56aa53ddfa8d405db14e40d13c45f344 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:47:44.092370 8145 raft_consensus.cc:729] T 0861cff26c754297aeef467f9d2bb05e P 56aa53ddfa8d405db14e40d13c45f344 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56aa53ddfa8d405db14e40d13c45f344, State: Initialized, Role: FOLLOWER
I20260807 08:47:44.092408 8145 consensus_queue.cc:260] T 0861cff26c754297aeef467f9d2bb05e P 56aa53ddfa8d405db14e40d13c45f344 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd480a8c763249ba8217c035bf4fc386" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 46375 } } peers { permanent_uuid: "56aa53ddfa8d405db14e40d13c45f344" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 35393 } } peers { permanent_uuid: "0f212e32e8044e8c9eb244945ad1c908" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 45951 } }
I20260807 08:47:44.092500 8145 ts_tablet_manager.cc:1428] T 0861cff26c754297aeef467f9d2bb05e P 56aa53ddfa8d405db14e40d13c45f344: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:44.092705 8144 tablet_bootstrap.cc:492] T 0861cff26c754297aeef467f9d2bb05e P 0f212e32e8044e8c9eb244945ad1c908: No bootstrap required, opened a new log
I20260807 08:47:44.092736 8146 tablet_bootstrap.cc:492] T 0861cff26c754297aeef467f9d2bb05e P fd480a8c763249ba8217c035bf4fc386: No bootstrap required, opened a new log
I20260807 08:47:44.092757 8144 ts_tablet_manager.cc:1397] T 0861cff26c754297aeef467f9d2bb05e P 0f212e32e8044e8c9eb244945ad1c908: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 08:47:44.092777 8146 ts_tablet_manager.cc:1397] T 0861cff26c754297aeef467f9d2bb05e P fd480a8c763249ba8217c035bf4fc386: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 08:47:44.092892 8146 raft_consensus.cc:348] T 0861cff26c754297aeef467f9d2bb05e P fd480a8c763249ba8217c035bf4fc386 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd480a8c763249ba8217c035bf4fc386" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 46375 } } peers { permanent_uuid: "56aa53ddfa8d405db14e40d13c45f344" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 35393 } } peers { permanent_uuid: "0f212e32e8044e8c9eb244945ad1c908" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 45951 } }
I20260807 08:47:44.092908 8144 raft_consensus.cc:348] T 0861cff26c754297aeef467f9d2bb05e P 0f212e32e8044e8c9eb244945ad1c908 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd480a8c763249ba8217c035bf4fc386" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 46375 } } peers { permanent_uuid: "56aa53ddfa8d405db14e40d13c45f344" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 35393 } } peers { permanent_uuid: "0f212e32e8044e8c9eb244945ad1c908" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 45951 } }
I20260807 08:47:44.092975 8144 raft_consensus.cc:374] T 0861cff26c754297aeef467f9d2bb05e P 0f212e32e8044e8c9eb244945ad1c908 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:47:44.092994 8144 raft_consensus.cc:729] T 0861cff26c754297aeef467f9d2bb05e P 0f212e32e8044e8c9eb244945ad1c908 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f212e32e8044e8c9eb244945ad1c908, State: Initialized, Role: FOLLOWER
I20260807 08:47:44.093056 8144 consensus_queue.cc:260] T 0861cff26c754297aeef467f9d2bb05e P 0f212e32e8044e8c9eb244945ad1c908 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd480a8c763249ba8217c035bf4fc386" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 46375 } } peers { permanent_uuid: "56aa53ddfa8d405db14e40d13c45f344" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 35393 } } peers { permanent_uuid: "0f212e32e8044e8c9eb244945ad1c908" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 45951 } }
I20260807 08:47:44.093114 8146 raft_consensus.cc:374] T 0861cff26c754297aeef467f9d2bb05e P fd480a8c763249ba8217c035bf4fc386 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:47:44.093200 8144 ts_tablet_manager.cc:1428] T 0861cff26c754297aeef467f9d2bb05e P 0f212e32e8044e8c9eb244945ad1c908: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:44.093343 8146 raft_consensus.cc:729] T 0861cff26c754297aeef467f9d2bb05e P fd480a8c763249ba8217c035bf4fc386 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd480a8c763249ba8217c035bf4fc386, State: Initialized, Role: FOLLOWER
I20260807 08:47:44.093415 8146 consensus_queue.cc:260] T 0861cff26c754297aeef467f9d2bb05e P fd480a8c763249ba8217c035bf4fc386 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd480a8c763249ba8217c035bf4fc386" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 46375 } } peers { permanent_uuid: "56aa53ddfa8d405db14e40d13c45f344" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 35393 } } peers { permanent_uuid: "0f212e32e8044e8c9eb244945ad1c908" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 45951 } }
I20260807 08:47:44.093588 8146 ts_tablet_manager.cc:1428] T 0861cff26c754297aeef467f9d2bb05e P fd480a8c763249ba8217c035bf4fc386: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
W20260807 08:47:44.114151 7598 tablet.cc:2369] T 0861cff26c754297aeef467f9d2bb05e P fd480a8c763249ba8217c035bf4fc386: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 08:47:44.205183 8035 raft_consensus.cc:482] T 0861cff26c754297aeef467f9d2bb05e P 0f212e32e8044e8c9eb244945ad1c908 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:47:44.205265 8035 raft_consensus.cc:504] T 0861cff26c754297aeef467f9d2bb05e P 0f212e32e8044e8c9eb244945ad1c908 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd480a8c763249ba8217c035bf4fc386" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 46375 } } peers { permanent_uuid: "56aa53ddfa8d405db14e40d13c45f344" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 35393 } } peers { permanent_uuid: "0f212e32e8044e8c9eb244945ad1c908" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 45951 } }
I20260807 08:47:44.205420 8035 leader_election.cc:288] T 0861cff26c754297aeef467f9d2bb05e P 0f212e32e8044e8c9eb244945ad1c908 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fd480a8c763249ba8217c035bf4fc386 (127.4.159.129:46375), 56aa53ddfa8d405db14e40d13c45f344 (127.4.159.130:35393)
I20260807 08:47:44.205605 7531 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0861cff26c754297aeef467f9d2bb05e" candidate_uuid: "0f212e32e8044e8c9eb244945ad1c908" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd480a8c763249ba8217c035bf4fc386" is_pre_election: true
I20260807 08:47:44.205672 7746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0861cff26c754297aeef467f9d2bb05e" candidate_uuid: "0f212e32e8044e8c9eb244945ad1c908" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56aa53ddfa8d405db14e40d13c45f344" is_pre_election: true
I20260807 08:47:44.205729 7531 raft_consensus.cc:2454] T 0861cff26c754297aeef467f9d2bb05e P fd480a8c763249ba8217c035bf4fc386 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f212e32e8044e8c9eb244945ad1c908 in term 0.
I20260807 08:47:44.205758 7746 raft_consensus.cc:2454] T 0861cff26c754297aeef467f9d2bb05e P 56aa53ddfa8d405db14e40d13c45f344 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f212e32e8044e8c9eb244945ad1c908 in term 0.
I20260807 08:47:44.205890 7838 leader_election.cc:302] T 0861cff26c754297aeef467f9d2bb05e P 0f212e32e8044e8c9eb244945ad1c908 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f212e32e8044e8c9eb244945ad1c908, fd480a8c763249ba8217c035bf4fc386; no voters:
I20260807 08:47:44.206017 8035 raft_consensus.cc:2781] T 0861cff26c754297aeef467f9d2bb05e P 0f212e32e8044e8c9eb244945ad1c908 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:47:44.206060 8035 raft_consensus.cc:482] T 0861cff26c754297aeef467f9d2bb05e P 0f212e32e8044e8c9eb244945ad1c908 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:47:44.206080 8035 raft_consensus.cc:3037] T 0861cff26c754297aeef467f9d2bb05e P 0f212e32e8044e8c9eb244945ad1c908 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:47:44.206771 8035 raft_consensus.cc:504] T 0861cff26c754297aeef467f9d2bb05e P 0f212e32e8044e8c9eb244945ad1c908 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd480a8c763249ba8217c035bf4fc386" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 46375 } } peers { permanent_uuid: "56aa53ddfa8d405db14e40d13c45f344" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 35393 } } peers { permanent_uuid: "0f212e32e8044e8c9eb244945ad1c908" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 45951 } }
I20260807 08:47:44.206892 8035 leader_election.cc:288] T 0861cff26c754297aeef467f9d2bb05e P 0f212e32e8044e8c9eb244945ad1c908 [CANDIDATE]: Term 1 election: Requested vote from peers fd480a8c763249ba8217c035bf4fc386 (127.4.159.129:46375), 56aa53ddfa8d405db14e40d13c45f344 (127.4.159.130:35393)
I20260807 08:47:44.207029 7531 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0861cff26c754297aeef467f9d2bb05e" candidate_uuid: "0f212e32e8044e8c9eb244945ad1c908" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd480a8c763249ba8217c035bf4fc386"
I20260807 08:47:44.207034 7746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0861cff26c754297aeef467f9d2bb05e" candidate_uuid: "0f212e32e8044e8c9eb244945ad1c908" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56aa53ddfa8d405db14e40d13c45f344"
I20260807 08:47:44.207118 7746 raft_consensus.cc:3037] T 0861cff26c754297aeef467f9d2bb05e P 56aa53ddfa8d405db14e40d13c45f344 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:47:44.207106 7531 raft_consensus.cc:3037] T 0861cff26c754297aeef467f9d2bb05e P fd480a8c763249ba8217c035bf4fc386 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:47:44.207681 7746 raft_consensus.cc:2454] T 0861cff26c754297aeef467f9d2bb05e P 56aa53ddfa8d405db14e40d13c45f344 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f212e32e8044e8c9eb244945ad1c908 in term 1.
I20260807 08:47:44.207803 7531 raft_consensus.cc:2454] T 0861cff26c754297aeef467f9d2bb05e P fd480a8c763249ba8217c035bf4fc386 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f212e32e8044e8c9eb244945ad1c908 in term 1.
I20260807 08:47:44.207826 7836 leader_election.cc:302] T 0861cff26c754297aeef467f9d2bb05e P 0f212e32e8044e8c9eb244945ad1c908 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f212e32e8044e8c9eb244945ad1c908, 56aa53ddfa8d405db14e40d13c45f344; no voters:
I20260807 08:47:44.208040 8035 raft_consensus.cc:2781] T 0861cff26c754297aeef467f9d2bb05e P 0f212e32e8044e8c9eb244945ad1c908 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:47:44.208092 8035 raft_consensus.cc:686] T 0861cff26c754297aeef467f9d2bb05e P 0f212e32e8044e8c9eb244945ad1c908 [term 1 LEADER]: Becoming Leader. State: Replica: 0f212e32e8044e8c9eb244945ad1c908, State: Running, Role: LEADER
I20260807 08:47:44.208133 8035 consensus_queue.cc:237] T 0861cff26c754297aeef467f9d2bb05e P 0f212e32e8044e8c9eb244945ad1c908 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd480a8c763249ba8217c035bf4fc386" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 46375 } } peers { permanent_uuid: "56aa53ddfa8d405db14e40d13c45f344" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 35393 } } peers { permanent_uuid: "0f212e32e8044e8c9eb244945ad1c908" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 45951 } }
I20260807 08:47:44.208678 7310 catalog_manager.cc:5495] T 0861cff26c754297aeef467f9d2bb05e P 0f212e32e8044e8c9eb244945ad1c908 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0f212e32e8044e8c9eb244945ad1c908 (127.4.159.131). New cstate: current_term: 1 leader_uuid: "0f212e32e8044e8c9eb244945ad1c908" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd480a8c763249ba8217c035bf4fc386" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 46375 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56aa53ddfa8d405db14e40d13c45f344" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 35393 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f212e32e8044e8c9eb244945ad1c908" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 45951 } health_report { overall_health: HEALTHY } } }
I20260807 08:47:44.237872 8155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:44.235672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42100 (local address 127.4.159.131:45951)
0807 08:47:44.235817 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:44.235819 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:44.235826 (+ 7us) server_negotiation.cc:407] Connection header received
0807 08:47:44.235845 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:44.235846 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:44.235855 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:44.235908 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:47:44.236047 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:44.236519 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:44.237248 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:44.237454 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:44.237507 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:44.237651 (+ 144us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:47:44.237685 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:47:44.237726 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:44.237779 (+ 53us) server_negotiation.cc:299] Negotiation successful
0807 08:47:44.237801 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":80,"threads_started":1}
I20260807 08:47:44.241927 8155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:44.239857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.131:52147 (local address 127.4.159.131:45951)
0807 08:47:44.239911 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:44.239913 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:44.240012 (+ 99us) server_negotiation.cc:407] Connection header received
0807 08:47:44.240068 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:44.240069 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:44.240078 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:44.240120 (+ 42us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:47:44.240210 (+ 90us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:44.240643 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:44.241327 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:44.241516 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:44.241567 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:44.241680 (+ 113us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:47:44.241708 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:47:44.241750 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:44.241802 (+ 52us) server_negotiation.cc:299] Negotiation successful
0807 08:47:44.241840 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":15,"server-negotiator.queue_time_us":35}
I20260807 08:47:44.241935 8159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:44.239814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.131:45951 (local address 127.4.159.131:52147)
0807 08:47:44.239948 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0807 08:47:44.239957 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 08:47:44.240017 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:47:44.240128 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:47:44.240129 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:47:44.240130 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 08:47:44.240168 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:44.240169 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:44.240711 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:44.240712 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:44.241260 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:44.241261 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:44.241574 (+ 313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:44.241575 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:44.241625 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:44.241628 (+ 3us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 08:47:44.241762 (+ 134us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 08:47:44.241763 (+ 1us) client_negotiation.cc:769] Sending connection context
0807 08:47:44.241802 (+ 39us) client_negotiation.cc:240] Negotiation successful
0807 08:47:44.241855 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"mutex_wait_us":2,"thread_start_us":95,"threads_started":1}
I20260807 08:47:44.243650 7746 raft_consensus.cc:1261] T 0861cff26c754297aeef467f9d2bb05e P 56aa53ddfa8d405db14e40d13c45f344 [term 1 FOLLOWER]: Refusing update from remote peer 0f212e32e8044e8c9eb244945ad1c908: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:47:44.243697 7531 raft_consensus.cc:1261] T 0861cff26c754297aeef467f9d2bb05e P fd480a8c763249ba8217c035bf4fc386 [term 1 FOLLOWER]: Refusing update from remote peer 0f212e32e8044e8c9eb244945ad1c908: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:47:44.243834 8105 consensus_queue.cc:1035] T 0861cff26c754297aeef467f9d2bb05e P 0f212e32e8044e8c9eb244945ad1c908 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56aa53ddfa8d405db14e40d13c45f344" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 35393 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:47:44.243920 8035 consensus_queue.cc:1035] T 0861cff26c754297aeef467f9d2bb05e P 0f212e32e8044e8c9eb244945ad1c908 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd480a8c763249ba8217c035bf4fc386" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 46375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:47:44.253304 7310 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51370:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:47:44.253378 7310 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51370:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:47:44.253903 7310 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d9cac9fdcacc45b693393f03561f7b11: Sending DeleteTablet for 3 replicas of tablet 0861cff26c754297aeef467f9d2bb05e
I20260807 08:47:44.254138 7505 tablet_service.cc:1514] Processing DeleteTablet for tablet 0861cff26c754297aeef467f9d2bb05e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:47:44 UTC) from {username='kudu', principal='kudu/127.4.159.190@KRBTEST.COM'} at 127.0.0.1:57780
I20260807 08:47:44.254223 8130 tablet_replica.cc:330] T 0861cff26c754297aeef467f9d2bb05e P fd480a8c763249ba8217c035bf4fc386: stopping tablet replica
I20260807 08:47:44.254204 7711 tablet_service.cc:1514] Processing DeleteTablet for tablet 0861cff26c754297aeef467f9d2bb05e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:47:44 UTC) from {username='kudu', principal='kudu/127.4.159.190@KRBTEST.COM'} at 127.0.0.1:55864
I20260807 08:47:44.254231 7904 tablet_service.cc:1514] Processing DeleteTablet for tablet 0861cff26c754297aeef467f9d2bb05e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:47:44 UTC) from {username='kudu', principal='kudu/127.4.159.190@KRBTEST.COM'} at 127.0.0.1:42090
I20260807 08:47:44.254263 8130 raft_consensus.cc:2229] T 0861cff26c754297aeef467f9d2bb05e P fd480a8c763249ba8217c035bf4fc386 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:47:44.254302 8130 raft_consensus.cc:2258] T 0861cff26c754297aeef467f9d2bb05e P fd480a8c763249ba8217c035bf4fc386 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:47:44.254320 8132 tablet_replica.cc:330] T 0861cff26c754297aeef467f9d2bb05e P 0f212e32e8044e8c9eb244945ad1c908: stopping tablet replica
I20260807 08:47:44.254331 8131 tablet_replica.cc:330] T 0861cff26c754297aeef467f9d2bb05e P 56aa53ddfa8d405db14e40d13c45f344: stopping tablet replica
I20260807 08:47:44.254390 8132 raft_consensus.cc:2229] T 0861cff26c754297aeef467f9d2bb05e P 0f212e32e8044e8c9eb244945ad1c908 [term 1 LEADER]: Raft consensus shutting down.
I20260807 08:47:44.254403 8131 raft_consensus.cc:2229] T 0861cff26c754297aeef467f9d2bb05e P 56aa53ddfa8d405db14e40d13c45f344 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:47:44.254436 8131 raft_consensus.cc:2258] T 0861cff26c754297aeef467f9d2bb05e P 56aa53ddfa8d405db14e40d13c45f344 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:47:44.254470 8132 raft_consensus.cc:2258] T 0861cff26c754297aeef467f9d2bb05e P 0f212e32e8044e8c9eb244945ad1c908 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:47:44.254737 8132 ts_tablet_manager.cc:1905] T 0861cff26c754297aeef467f9d2bb05e P 0f212e32e8044e8c9eb244945ad1c908: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:47:44.254886 8131 ts_tablet_manager.cc:1905] T 0861cff26c754297aeef467f9d2bb05e P 56aa53ddfa8d405db14e40d13c45f344: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:47:44.255008 8130 ts_tablet_manager.cc:1905] T 0861cff26c754297aeef467f9d2bb05e P fd480a8c763249ba8217c035bf4fc386: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:47:44.256381 8131 ts_tablet_manager.cc:1918] T 0861cff26c754297aeef467f9d2bb05e P 56aa53ddfa8d405db14e40d13c45f344: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 08:47:44.256390 8130 ts_tablet_manager.cc:1918] T 0861cff26c754297aeef467f9d2bb05e P fd480a8c763249ba8217c035bf4fc386: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 08:47:44.256430 8130 log.cc:1196] T 0861cff26c754297aeef467f9d2bb05e P fd480a8c763249ba8217c035bf4fc386: Deleting WAL directory at /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786092453486433-4734-0/minicluster-data/ts-0/wal/wals/0861cff26c754297aeef467f9d2bb05e
I20260807 08:47:44.256433 8131 log.cc:1196] T 0861cff26c754297aeef467f9d2bb05e P 56aa53ddfa8d405db14e40d13c45f344: Deleting WAL directory at /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786092453486433-4734-0/minicluster-data/ts-1/wal/wals/0861cff26c754297aeef467f9d2bb05e
I20260807 08:47:44.256587 8130 ts_tablet_manager.cc:1939] T 0861cff26c754297aeef467f9d2bb05e P fd480a8c763249ba8217c035bf4fc386: Deleting consensus metadata
I20260807 08:47:44.256633 8131 ts_tablet_manager.cc:1939] T 0861cff26c754297aeef467f9d2bb05e P 56aa53ddfa8d405db14e40d13c45f344: Deleting consensus metadata
I20260807 08:47:44.256853 8132 ts_tablet_manager.cc:1918] T 0861cff26c754297aeef467f9d2bb05e P 0f212e32e8044e8c9eb244945ad1c908: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 08:47:44.256875 7292 catalog_manager.cc:4841] TS fd480a8c763249ba8217c035bf4fc386 (127.4.159.129:46375): tablet 0861cff26c754297aeef467f9d2bb05e (table test-table [id=2276d9b839164e41b5d4662b8c1f73c1]) successfully deleted
I20260807 08:47:44.256901 8132 log.cc:1196] T 0861cff26c754297aeef467f9d2bb05e P 0f212e32e8044e8c9eb244945ad1c908: Deleting WAL directory at /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786092453486433-4734-0/minicluster-data/ts-2/wal/wals/0861cff26c754297aeef467f9d2bb05e
I20260807 08:47:44.257059 8132 ts_tablet_manager.cc:1939] T 0861cff26c754297aeef467f9d2bb05e P 0f212e32e8044e8c9eb244945ad1c908: Deleting consensus metadata
I20260807 08:47:44.257083 7289 catalog_manager.cc:4841] TS 56aa53ddfa8d405db14e40d13c45f344 (127.4.159.130:35393): tablet 0861cff26c754297aeef467f9d2bb05e (table test-table [id=2276d9b839164e41b5d4662b8c1f73c1]) successfully deleted
I20260807 08:47:44.257314 7310 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51370:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 08:47:44.257452 7310 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:47:44.257597 7289 catalog_manager.cc:4841] TS 0f212e32e8044e8c9eb244945ad1c908 (127.4.159.131:45951): tablet 0861cff26c754297aeef467f9d2bb05e (table test-table [id=2276d9b839164e41b5d4662b8c1f73c1]) successfully deleted
I20260807 08:47:44.258515 7904 tablet_service.cc:1467] Processing CreateTablet for tablet 72f3a0b2bb474c97a127f61183525571 (DEFAULT_TABLE table=test-table [id=22e1269440e04911a119f4c71a2ec1d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:47:44.258515 7505 tablet_service.cc:1467] Processing CreateTablet for tablet 72f3a0b2bb474c97a127f61183525571 (DEFAULT_TABLE table=test-table [id=22e1269440e04911a119f4c71a2ec1d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:47:44.258522 7711 tablet_service.cc:1467] Processing CreateTablet for tablet 72f3a0b2bb474c97a127f61183525571 (DEFAULT_TABLE table=test-table [id=22e1269440e04911a119f4c71a2ec1d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:47:44.258621 7711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 72f3a0b2bb474c97a127f61183525571. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:47:44.258621 7904 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 72f3a0b2bb474c97a127f61183525571. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:47:44.258620 7505 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 72f3a0b2bb474c97a127f61183525571. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:47:44.259719 8145 tablet_bootstrap.cc:492] T 72f3a0b2bb474c97a127f61183525571 P 56aa53ddfa8d405db14e40d13c45f344: Bootstrap starting.
I20260807 08:47:44.259730 8144 tablet_bootstrap.cc:492] T 72f3a0b2bb474c97a127f61183525571 P 0f212e32e8044e8c9eb244945ad1c908: Bootstrap starting.
I20260807 08:47:44.259739 8146 tablet_bootstrap.cc:492] T 72f3a0b2bb474c97a127f61183525571 P fd480a8c763249ba8217c035bf4fc386: Bootstrap starting.
I20260807 08:47:44.260001 8145 tablet_bootstrap.cc:654] T 72f3a0b2bb474c97a127f61183525571 P 56aa53ddfa8d405db14e40d13c45f344: Neither blocks nor log segments found. Creating new log.
I20260807 08:47:44.260047 8144 tablet_bootstrap.cc:654] T 72f3a0b2bb474c97a127f61183525571 P 0f212e32e8044e8c9eb244945ad1c908: Neither blocks nor log segments found. Creating new log.
I20260807 08:47:44.260249 8146 tablet_bootstrap.cc:654] T 72f3a0b2bb474c97a127f61183525571 P fd480a8c763249ba8217c035bf4fc386: Neither blocks nor log segments found. Creating new log.
I20260807 08:47:44.260682 8144 tablet_bootstrap.cc:492] T 72f3a0b2bb474c97a127f61183525571 P 0f212e32e8044e8c9eb244945ad1c908: No bootstrap required, opened a new log
I20260807 08:47:44.260728 8144 ts_tablet_manager.cc:1397] T 72f3a0b2bb474c97a127f61183525571 P 0f212e32e8044e8c9eb244945ad1c908: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 08:47:44.260793 8146 tablet_bootstrap.cc:492] T 72f3a0b2bb474c97a127f61183525571 P fd480a8c763249ba8217c035bf4fc386: No bootstrap required, opened a new log
I20260807 08:47:44.260835 8146 ts_tablet_manager.cc:1397] T 72f3a0b2bb474c97a127f61183525571 P fd480a8c763249ba8217c035bf4fc386: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 08:47:44.260864 8144 raft_consensus.cc:348] T 72f3a0b2bb474c97a127f61183525571 P 0f212e32e8044e8c9eb244945ad1c908 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd480a8c763249ba8217c035bf4fc386" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 46375 } } peers { permanent_uuid: "56aa53ddfa8d405db14e40d13c45f344" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 35393 } } peers { permanent_uuid: "0f212e32e8044e8c9eb244945ad1c908" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 45951 } }
I20260807 08:47:44.260921 8144 raft_consensus.cc:374] T 72f3a0b2bb474c97a127f61183525571 P 0f212e32e8044e8c9eb244945ad1c908 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:47:44.260916 8145 tablet_bootstrap.cc:492] T 72f3a0b2bb474c97a127f61183525571 P 56aa53ddfa8d405db14e40d13c45f344: No bootstrap required, opened a new log
I20260807 08:47:44.260939 8144 raft_consensus.cc:729] T 72f3a0b2bb474c97a127f61183525571 P 0f212e32e8044e8c9eb244945ad1c908 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f212e32e8044e8c9eb244945ad1c908, State: Initialized, Role: FOLLOWER
I20260807 08:47:44.260954 8145 ts_tablet_manager.cc:1397] T 72f3a0b2bb474c97a127f61183525571 P 56aa53ddfa8d405db14e40d13c45f344: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260807 08:47:44.260949 8146 raft_consensus.cc:348] T 72f3a0b2bb474c97a127f61183525571 P fd480a8c763249ba8217c035bf4fc386 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd480a8c763249ba8217c035bf4fc386" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 46375 } } peers { permanent_uuid: "56aa53ddfa8d405db14e40d13c45f344" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 35393 } } peers { permanent_uuid: "0f212e32e8044e8c9eb244945ad1c908" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 45951 } }
I20260807 08:47:44.260977 8144 consensus_queue.cc:260] T 72f3a0b2bb474c97a127f61183525571 P 0f212e32e8044e8c9eb244945ad1c908 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd480a8c763249ba8217c035bf4fc386" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 46375 } } peers { permanent_uuid: "56aa53ddfa8d405db14e40d13c45f344" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 35393 } } peers { permanent_uuid: "0f212e32e8044e8c9eb244945ad1c908" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 45951 } }
I20260807 08:47:44.261005 8146 raft_consensus.cc:374] T 72f3a0b2bb474c97a127f61183525571 P fd480a8c763249ba8217c035bf4fc386 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:47:44.261023 8146 raft_consensus.cc:729] T 72f3a0b2bb474c97a127f61183525571 P fd480a8c763249ba8217c035bf4fc386 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd480a8c763249ba8217c035bf4fc386, State: Initialized, Role: FOLLOWER
I20260807 08:47:44.261054 8144 ts_tablet_manager.cc:1428] T 72f3a0b2bb474c97a127f61183525571 P 0f212e32e8044e8c9eb244945ad1c908: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:44.261065 8146 consensus_queue.cc:260] T 72f3a0b2bb474c97a127f61183525571 P fd480a8c763249ba8217c035bf4fc386 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd480a8c763249ba8217c035bf4fc386" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 46375 } } peers { permanent_uuid: "56aa53ddfa8d405db14e40d13c45f344" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 35393 } } peers { permanent_uuid: "0f212e32e8044e8c9eb244945ad1c908" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 45951 } }
I20260807 08:47:44.261104 8145 raft_consensus.cc:348] T 72f3a0b2bb474c97a127f61183525571 P 56aa53ddfa8d405db14e40d13c45f344 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd480a8c763249ba8217c035bf4fc386" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 46375 } } peers { permanent_uuid: "56aa53ddfa8d405db14e40d13c45f344" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 35393 } } peers { permanent_uuid: "0f212e32e8044e8c9eb244945ad1c908" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 45951 } }
I20260807 08:47:44.261163 8145 raft_consensus.cc:374] T 72f3a0b2bb474c97a127f61183525571 P 56aa53ddfa8d405db14e40d13c45f344 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:47:44.261181 8145 raft_consensus.cc:729] T 72f3a0b2bb474c97a127f61183525571 P 56aa53ddfa8d405db14e40d13c45f344 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56aa53ddfa8d405db14e40d13c45f344, State: Initialized, Role: FOLLOWER
I20260807 08:47:44.261176 8146 ts_tablet_manager.cc:1428] T 72f3a0b2bb474c97a127f61183525571 P fd480a8c763249ba8217c035bf4fc386: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:44.261220 8145 consensus_queue.cc:260] T 72f3a0b2bb474c97a127f61183525571 P 56aa53ddfa8d405db14e40d13c45f344 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd480a8c763249ba8217c035bf4fc386" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 46375 } } peers { permanent_uuid: "56aa53ddfa8d405db14e40d13c45f344" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 35393 } } peers { permanent_uuid: "0f212e32e8044e8c9eb244945ad1c908" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 45951 } }
I20260807 08:47:44.261302 8145 ts_tablet_manager.cc:1428] T 72f3a0b2bb474c97a127f61183525571 P 56aa53ddfa8d405db14e40d13c45f344: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:44.352723 8102 raft_consensus.cc:482] T 72f3a0b2bb474c97a127f61183525571 P 56aa53ddfa8d405db14e40d13c45f344 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:47:44.352825 8102 raft_consensus.cc:504] T 72f3a0b2bb474c97a127f61183525571 P 56aa53ddfa8d405db14e40d13c45f344 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd480a8c763249ba8217c035bf4fc386" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 46375 } } peers { permanent_uuid: "56aa53ddfa8d405db14e40d13c45f344" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 35393 } } peers { permanent_uuid: "0f212e32e8044e8c9eb244945ad1c908" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 45951 } }
I20260807 08:47:44.352991 8102 leader_election.cc:288] T 72f3a0b2bb474c97a127f61183525571 P 56aa53ddfa8d405db14e40d13c45f344 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fd480a8c763249ba8217c035bf4fc386 (127.4.159.129:46375), 0f212e32e8044e8c9eb244945ad1c908 (127.4.159.131:45951)
I20260807 08:47:44.353219 7935 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72f3a0b2bb474c97a127f61183525571" candidate_uuid: "56aa53ddfa8d405db14e40d13c45f344" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f212e32e8044e8c9eb244945ad1c908" is_pre_election: true
I20260807 08:47:44.353219 7531 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72f3a0b2bb474c97a127f61183525571" candidate_uuid: "56aa53ddfa8d405db14e40d13c45f344" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd480a8c763249ba8217c035bf4fc386" is_pre_election: true
I20260807 08:47:44.353307 7935 raft_consensus.cc:2454] T 72f3a0b2bb474c97a127f61183525571 P 0f212e32e8044e8c9eb244945ad1c908 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56aa53ddfa8d405db14e40d13c45f344 in term 0.
I20260807 08:47:44.353320 7531 raft_consensus.cc:2454] T 72f3a0b2bb474c97a127f61183525571 P fd480a8c763249ba8217c035bf4fc386 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56aa53ddfa8d405db14e40d13c45f344 in term 0.
I20260807 08:47:44.353478 7630 leader_election.cc:302] T 72f3a0b2bb474c97a127f61183525571 P 56aa53ddfa8d405db14e40d13c45f344 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f212e32e8044e8c9eb244945ad1c908, 56aa53ddfa8d405db14e40d13c45f344; no voters:
I20260807 08:47:44.353597 8102 raft_consensus.cc:2781] T 72f3a0b2bb474c97a127f61183525571 P 56aa53ddfa8d405db14e40d13c45f344 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:47:44.353649 8102 raft_consensus.cc:482] T 72f3a0b2bb474c97a127f61183525571 P 56aa53ddfa8d405db14e40d13c45f344 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:47:44.353670 8102 raft_consensus.cc:3037] T 72f3a0b2bb474c97a127f61183525571 P 56aa53ddfa8d405db14e40d13c45f344 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:47:44.354409 8102 raft_consensus.cc:504] T 72f3a0b2bb474c97a127f61183525571 P 56aa53ddfa8d405db14e40d13c45f344 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd480a8c763249ba8217c035bf4fc386" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 46375 } } peers { permanent_uuid: "56aa53ddfa8d405db14e40d13c45f344" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 35393 } } peers { permanent_uuid: "0f212e32e8044e8c9eb244945ad1c908" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 45951 } }
I20260807 08:47:44.354542 8102 leader_election.cc:288] T 72f3a0b2bb474c97a127f61183525571 P 56aa53ddfa8d405db14e40d13c45f344 [CANDIDATE]: Term 1 election: Requested vote from peers fd480a8c763249ba8217c035bf4fc386 (127.4.159.129:46375), 0f212e32e8044e8c9eb244945ad1c908 (127.4.159.131:45951)
I20260807 08:47:44.354693 7935 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72f3a0b2bb474c97a127f61183525571" candidate_uuid: "56aa53ddfa8d405db14e40d13c45f344" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f212e32e8044e8c9eb244945ad1c908"
I20260807 08:47:44.354684 7531 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72f3a0b2bb474c97a127f61183525571" candidate_uuid: "56aa53ddfa8d405db14e40d13c45f344" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd480a8c763249ba8217c035bf4fc386"
I20260807 08:47:44.354761 7935 raft_consensus.cc:3037] T 72f3a0b2bb474c97a127f61183525571 P 0f212e32e8044e8c9eb244945ad1c908 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:47:44.354763 7531 raft_consensus.cc:3037] T 72f3a0b2bb474c97a127f61183525571 P fd480a8c763249ba8217c035bf4fc386 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:47:44.355340 7935 raft_consensus.cc:2454] T 72f3a0b2bb474c97a127f61183525571 P 0f212e32e8044e8c9eb244945ad1c908 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56aa53ddfa8d405db14e40d13c45f344 in term 1.
I20260807 08:47:44.355343 7531 raft_consensus.cc:2454] T 72f3a0b2bb474c97a127f61183525571 P fd480a8c763249ba8217c035bf4fc386 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56aa53ddfa8d405db14e40d13c45f344 in term 1.
I20260807 08:47:44.355507 7630 leader_election.cc:302] T 72f3a0b2bb474c97a127f61183525571 P 56aa53ddfa8d405db14e40d13c45f344 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f212e32e8044e8c9eb244945ad1c908, 56aa53ddfa8d405db14e40d13c45f344; no voters:
I20260807 08:47:44.355595 8102 raft_consensus.cc:2781] T 72f3a0b2bb474c97a127f61183525571 P 56aa53ddfa8d405db14e40d13c45f344 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:47:44.355667 8102 raft_consensus.cc:686] T 72f3a0b2bb474c97a127f61183525571 P 56aa53ddfa8d405db14e40d13c45f344 [term 1 LEADER]: Becoming Leader. State: Replica: 56aa53ddfa8d405db14e40d13c45f344, State: Running, Role: LEADER
I20260807 08:47:44.355726 8102 consensus_queue.cc:237] T 72f3a0b2bb474c97a127f61183525571 P 56aa53ddfa8d405db14e40d13c45f344 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd480a8c763249ba8217c035bf4fc386" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 46375 } } peers { permanent_uuid: "56aa53ddfa8d405db14e40d13c45f344" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 35393 } } peers { permanent_uuid: "0f212e32e8044e8c9eb244945ad1c908" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 45951 } }
I20260807 08:47:44.356307 7310 catalog_manager.cc:5495] T 72f3a0b2bb474c97a127f61183525571 P 56aa53ddfa8d405db14e40d13c45f344 reported cstate change: term changed from 0 to 1, leader changed from <none> to 56aa53ddfa8d405db14e40d13c45f344 (127.4.159.130). New cstate: current_term: 1 leader_uuid: "56aa53ddfa8d405db14e40d13c45f344" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd480a8c763249ba8217c035bf4fc386" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 46375 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56aa53ddfa8d405db14e40d13c45f344" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 35393 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f212e32e8044e8c9eb244945ad1c908" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 45951 } health_report { overall_health: UNKNOWN } } }
I20260807 08:47:44.379482 8170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:44.377458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55884 (local address 127.4.159.130:35393)
0807 08:47:44.377619 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:44.377621 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:44.377627 (+ 6us) server_negotiation.cc:407] Connection header received
0807 08:47:44.377645 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:44.377646 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:44.377655 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:44.377708 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:47:44.377819 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:44.378286 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:44.378983 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:44.379172 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:44.379214 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:44.379323 (+ 109us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:47:44.379349 (+ 26us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:47:44.379379 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:44.379411 (+ 32us) server_negotiation.cc:299] Negotiation successful
0807 08:47:44.379426 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":97,"threads_started":1}
I20260807 08:47:44.381500 7935 raft_consensus.cc:1261] T 72f3a0b2bb474c97a127f61183525571 P 0f212e32e8044e8c9eb244945ad1c908 [term 1 FOLLOWER]: Refusing update from remote peer 56aa53ddfa8d405db14e40d13c45f344: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:47:44.381512 7531 raft_consensus.cc:1261] T 72f3a0b2bb474c97a127f61183525571 P fd480a8c763249ba8217c035bf4fc386 [term 1 FOLLOWER]: Refusing update from remote peer 56aa53ddfa8d405db14e40d13c45f344: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:47:44.381726 8102 consensus_queue.cc:1035] T 72f3a0b2bb474c97a127f61183525571 P 56aa53ddfa8d405db14e40d13c45f344 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd480a8c763249ba8217c035bf4fc386" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 46375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:47:44.381831 8118 consensus_queue.cc:1035] T 72f3a0b2bb474c97a127f61183525571 P 56aa53ddfa8d405db14e40d13c45f344 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f212e32e8044e8c9eb244945ad1c908" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 45951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 08:47:44.385743 8107 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260807 08:47:45.901562 7309 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51370:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:47:45.901662 7309 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51370:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:47:45.902444 7309 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d9cac9fdcacc45b693393f03561f7b11: Sending DeleteTablet for 3 replicas of tablet 72f3a0b2bb474c97a127f61183525571
I20260807 08:47:45.902683 7505 tablet_service.cc:1514] Processing DeleteTablet for tablet 72f3a0b2bb474c97a127f61183525571 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:47:45 UTC) from {username='kudu', principal='kudu/127.4.159.190@KRBTEST.COM'} at 127.0.0.1:57780
I20260807 08:47:45.902741 7711 tablet_service.cc:1514] Processing DeleteTablet for tablet 72f3a0b2bb474c97a127f61183525571 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:47:45 UTC) from {username='kudu', principal='kudu/127.4.159.190@KRBTEST.COM'} at 127.0.0.1:55864
I20260807 08:47:45.902848 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 7413
I20260807 08:47:45.902933 7904 tablet_service.cc:1514] Processing DeleteTablet for tablet 72f3a0b2bb474c97a127f61183525571 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:47:45 UTC) from {username='kudu', principal='kudu/127.4.159.190@KRBTEST.COM'} at 127.0.0.1:42090
I20260807 08:47:45.910856 8212 tablet_replica.cc:330] T 72f3a0b2bb474c97a127f61183525571 P 0f212e32e8044e8c9eb244945ad1c908: stopping tablet replica
I20260807 08:47:45.910961 8212 raft_consensus.cc:2229] T 72f3a0b2bb474c97a127f61183525571 P 0f212e32e8044e8c9eb244945ad1c908 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:47:45.911011 8212 raft_consensus.cc:2258] T 72f3a0b2bb474c97a127f61183525571 P 0f212e32e8044e8c9eb244945ad1c908 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:47:45.911247 8212 ts_tablet_manager.cc:1905] T 72f3a0b2bb474c97a127f61183525571 P 0f212e32e8044e8c9eb244945ad1c908: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:47:45.912840 8212 ts_tablet_manager.cc:1918] T 72f3a0b2bb474c97a127f61183525571 P 0f212e32e8044e8c9eb244945ad1c908: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 08:47:45.912904 8212 log.cc:1196] T 72f3a0b2bb474c97a127f61183525571 P 0f212e32e8044e8c9eb244945ad1c908: Deleting WAL directory at /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786092453486433-4734-0/minicluster-data/ts-2/wal/wals/72f3a0b2bb474c97a127f61183525571
I20260807 08:47:45.913100 8212 ts_tablet_manager.cc:1939] T 72f3a0b2bb474c97a127f61183525571 P 0f212e32e8044e8c9eb244945ad1c908: Deleting consensus metadata
I20260807 08:47:45.913370 8211 tablet_replica.cc:330] T 72f3a0b2bb474c97a127f61183525571 P 56aa53ddfa8d405db14e40d13c45f344: stopping tablet replica
I20260807 08:47:45.913404 7289 catalog_manager.cc:4841] TS 0f212e32e8044e8c9eb244945ad1c908 (127.4.159.131:45951): tablet 72f3a0b2bb474c97a127f61183525571 (table test-table [id=22e1269440e04911a119f4c71a2ec1d4]) successfully deleted
I20260807 08:47:45.913472 8211 raft_consensus.cc:2229] T 72f3a0b2bb474c97a127f61183525571 P 56aa53ddfa8d405db14e40d13c45f344 [term 1 LEADER]: Raft consensus shutting down.
I20260807 08:47:45.913528 8211 raft_consensus.cc:2258] T 72f3a0b2bb474c97a127f61183525571 P 56aa53ddfa8d405db14e40d13c45f344 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:47:45.913757 8211 ts_tablet_manager.cc:1905] T 72f3a0b2bb474c97a127f61183525571 P 56aa53ddfa8d405db14e40d13c45f344: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:47:45.915261 8211 ts_tablet_manager.cc:1918] T 72f3a0b2bb474c97a127f61183525571 P 56aa53ddfa8d405db14e40d13c45f344: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 08:47:45.915319 8211 log.cc:1196] T 72f3a0b2bb474c97a127f61183525571 P 56aa53ddfa8d405db14e40d13c45f344: Deleting WAL directory at /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786092453486433-4734-0/minicluster-data/ts-1/wal/wals/72f3a0b2bb474c97a127f61183525571
I20260807 08:47:45.915541 8211 ts_tablet_manager.cc:1939] T 72f3a0b2bb474c97a127f61183525571 P 56aa53ddfa8d405db14e40d13c45f344: Deleting consensus metadata
I20260807 08:47:45.915843 7289 catalog_manager.cc:4841] TS 56aa53ddfa8d405db14e40d13c45f344 (127.4.159.130:35393): tablet 72f3a0b2bb474c97a127f61183525571 (table test-table [id=22e1269440e04911a119f4c71a2ec1d4]) successfully deleted
W20260807 08:47:45.916204 7292 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.159.129:46375 (error 108)
I20260807 08:47:45.916491 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 7608
I20260807 08:47:45.920010 8214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:45.916554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.129:46375 (local address 127.0.0.1:56350)
0807 08:47:45.919932 (+ 3378us) negotiation.cc:107] Waiting for socket to connect
0807 08:47:45.919978 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.129:46375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3356,"thread_start_us":59,"threads_started":1}
W20260807 08:47:45.920126 7292 catalog_manager.cc:4584] TS fd480a8c763249ba8217c035bf4fc386 (127.4.159.129:46375): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 72f3a0b2bb474c97a127f61183525571: Network error: Client connection negotiation failed: client connection to 127.4.159.129:46375: connect: Connection refused (error 111)
I20260807 08:47:45.925931 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 7817
I20260807 08:47:45.933393 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 7261
2026-08-07T08:47:45Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (5604 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786092453486433-4734-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:47:45 dist-test-slave-6zbq krb5kdc[8218](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:47:45 dist-test-slave-6zbq krb5kdc[8218](info): set up 2 sockets
Aug 07 08:47:45 dist-test-slave-6zbq krb5kdc[8218](info): commencing operation
krb5kdc: starting...
W20260807 08:47:47.041397 4734 mini_kdc.cc:121] Time spent starting KDC: real 1.085s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786092453486433-4734-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786092453486433-4734-0/krb5kdc/test-user.keytab.
Aug 07 08:47:47 dist-test-slave-6zbq krb5kdc[8218](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092467, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:47:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:47:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.159.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.159.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.159.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab.
Entry for principal kudu/127.4.159.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab.
I20260807 08:47:47.145653 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786092453486433-4734-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786092453486433-4734-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786092453486433-4734-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.159.190:43537
--webserver_interface=127.4.159.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab
--principal=kudu/127.4.159.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:36675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.159.190:43537
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:47:47.224334 8232 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:47:47.224519 8232 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:47:47.224540 8232 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:47:47.225651 8232 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:47:47.225692 8232 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:47:47.225706 8232 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:47:47.225729 8232 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:47:47.225751 8232 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:47:47.227221 8232 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:36675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786092453486433-4734-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786092453486433-4734-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.159.190:43537
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab
--principal=kudu/127.4.159.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.159.190:43537
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.159.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8232
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786092453486433-4734-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:47:47.227444 8232 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:47:47.227643 8232 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:47:47.232035 8240 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:47:47.232017 8239 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:47:47.232016 8242 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:47:47.232128 8232 server_base.cc:1034] running on GCE node
I20260807 08:47:47.232533 8232 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:47:47.232800 8232 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:47:47.233943 8232 hybrid_clock.cc:648] HybridClock initialized: now 1786092467233929 us; error 34 us; skew 500 ppm
Aug 07 08:47:47 dist-test-slave-6zbq krb5kdc[8218](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092467, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:47:47.236203 8232 init.cc:377] Logged in from keytab as kudu/127.4.159.190@KRBTEST.COM (short username kudu)
I20260807 08:47:47.236562 8232 webserver.cc:460] Webserver started at http://127.4.159.190:46277/ using document root <none> and password file <none>
I20260807 08:47:47.236734 8232 fs_manager.cc:359] Metadata directory not provided
I20260807 08:47:47.236773 8232 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:47:47.236868 8232 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:47:47.237787 8232 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786092453486433-4734-0/minicluster-data/master-0/data/instance:
uuid: "935be513e068495c9d751b758fba8f87"
format_stamp: "Formatted at 2026-08-07 08:47:47 on dist-test-slave-6zbq"
server_key: "1394ef1151d6a100a632cbcc7fdd79c6"
server_key_iv: "8877b511b24acb468d3eb5159d4d1b8e"
server_key_version: "encryptionkey@0"
I20260807 08:47:47.238083 8232 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786092453486433-4734-0/minicluster-data/master-0/wal/instance:
uuid: "935be513e068495c9d751b758fba8f87"
format_stamp: "Formatted at 2026-08-07 08:47:47 on dist-test-slave-6zbq"
server_key: "1394ef1151d6a100a632cbcc7fdd79c6"
server_key_iv: "8877b511b24acb468d3eb5159d4d1b8e"
server_key_version: "encryptionkey@0"
I20260807 08:47:47.239323 8232 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 08:47:47.239918 8249 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:47.240044 8232 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 08:47:47.240113 8232 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786092453486433-4734-0/minicluster-data/master-0/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786092453486433-4734-0/minicluster-data/master-0/wal
uuid: "935be513e068495c9d751b758fba8f87"
format_stamp: "Formatted at 2026-08-07 08:47:47 on dist-test-slave-6zbq"
server_key: "1394ef1151d6a100a632cbcc7fdd79c6"
server_key_iv: "8877b511b24acb468d3eb5159d4d1b8e"
server_key_version: "encryptionkey@0"
I20260807 08:47:47.240165 8232 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786092453486433-4734-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786092453486433-4734-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786092453486433-4734-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:47:47.257301 8232 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:47:47.259265 8232 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:47:47.259403 8232 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:47:47.263065 8232 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.190:43537
I20260807 08:47:47.263113 8336 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.190:43537 every 8 connection(s)
I20260807 08:47:47.263427 8232 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
I20260807 08:47:47.264258 8337 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:47:47.266420 8337 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 935be513e068495c9d751b758fba8f87: Bootstrap starting.
I20260807 08:47:47.266958 8337 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 935be513e068495c9d751b758fba8f87: Neither blocks nor log segments found. Creating new log.
I20260807 08:47:47.267216 8337 log.cc:824] T 00000000000000000000000000000000 P 935be513e068495c9d751b758fba8f87: Log is configured to *not* fsync() on all Append() calls
I20260807 08:47:47.267834 8337 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 935be513e068495c9d751b758fba8f87: No bootstrap required, opened a new log
I20260807 08:47:47.269373 8337 raft_consensus.cc:348] T 00000000000000000000000000000000 P 935be513e068495c9d751b758fba8f87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "935be513e068495c9d751b758fba8f87" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 43537 } }
I20260807 08:47:47.269505 8337 raft_consensus.cc:374] T 00000000000000000000000000000000 P 935be513e068495c9d751b758fba8f87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:47:47.269537 8337 raft_consensus.cc:729] T 00000000000000000000000000000000 P 935be513e068495c9d751b758fba8f87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 935be513e068495c9d751b758fba8f87, State: Initialized, Role: FOLLOWER
I20260807 08:47:47.269553 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 8232
I20260807 08:47:47.269658 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786092453486433-4734-0/minicluster-data/master-0/wal/instance
I20260807 08:47:47.269675 8337 consensus_queue.cc:260] T 00000000000000000000000000000000 P 935be513e068495c9d751b758fba8f87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "935be513e068495c9d751b758fba8f87" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 43537 } }
I20260807 08:47:47.269760 8337 raft_consensus.cc:388] T 00000000000000000000000000000000 P 935be513e068495c9d751b758fba8f87 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:47:47.269797 8337 raft_consensus.cc:482] T 00000000000000000000000000000000 P 935be513e068495c9d751b758fba8f87 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:47:47.269830 4734 external_mini_cluster.cc:1428] Setting key 39bec53b7bfc8b2a8c18e1e655f753ec
I20260807 08:47:47.269909 8337 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 935be513e068495c9d751b758fba8f87 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:47:47.270622 8337 raft_consensus.cc:504] T 00000000000000000000000000000000 P 935be513e068495c9d751b758fba8f87 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "935be513e068495c9d751b758fba8f87" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 43537 } }
I20260807 08:47:47.270726 8337 leader_election.cc:302] T 00000000000000000000000000000000 P 935be513e068495c9d751b758fba8f87 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 935be513e068495c9d751b758fba8f87; no voters:
I20260807 08:47:47.270886 8337 leader_election.cc:288] T 00000000000000000000000000000000 P 935be513e068495c9d751b758fba8f87 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:47:47.270936 8344 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 935be513e068495c9d751b758fba8f87 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:47:47.271075 8344 raft_consensus.cc:686] T 00000000000000000000000000000000 P 935be513e068495c9d751b758fba8f87 [term 1 LEADER]: Becoming Leader. State: Replica: 935be513e068495c9d751b758fba8f87, State: Running, Role: LEADER
I20260807 08:47:47.271162 8337 sys_catalog.cc:564] T 00000000000000000000000000000000 P 935be513e068495c9d751b758fba8f87 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:47:47.271184 8344 consensus_queue.cc:237] T 00000000000000000000000000000000 P 935be513e068495c9d751b758fba8f87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "935be513e068495c9d751b758fba8f87" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 43537 } }
I20260807 08:47:47.271718 8346 sys_catalog.cc:455] T 00000000000000000000000000000000 P 935be513e068495c9d751b758fba8f87 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 935be513e068495c9d751b758fba8f87. Latest consensus state: current_term: 1 leader_uuid: "935be513e068495c9d751b758fba8f87" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "935be513e068495c9d751b758fba8f87" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 43537 } } }
I20260807 08:47:47.271807 8346 sys_catalog.cc:458] T 00000000000000000000000000000000 P 935be513e068495c9d751b758fba8f87 [sys.catalog]: This master's current role is: LEADER
I20260807 08:47:47.272018 8345 sys_catalog.cc:455] T 00000000000000000000000000000000 P 935be513e068495c9d751b758fba8f87 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "935be513e068495c9d751b758fba8f87" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "935be513e068495c9d751b758fba8f87" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 43537 } } }
I20260807 08:47:47.272082 8345 sys_catalog.cc:458] T 00000000000000000000000000000000 P 935be513e068495c9d751b758fba8f87 [sys.catalog]: This master's current role is: LEADER
I20260807 08:47:47.272437 8354 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:47:47.272914 8354 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:47:47.274590 8354 catalog_manager.cc:1347] Generated new cluster ID: e84c6c2e2b134dccbb87a41128baa2c7
I20260807 08:47:47.274644 8354 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 08:47:47 dist-test-slave-6zbq krb5kdc[8218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092467, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.159.190@KRBTEST.COM
I20260807 08:47:47.282172 8341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:47.270287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47540 (local address 127.4.159.190:43537)
0807 08:47:47.270744 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:47.270754 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:47.270785 (+ 31us) server_negotiation.cc:407] Connection header received
0807 08:47:47.270894 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:47.270899 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:47.270945 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:47.271853 (+ 908us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:47:47.271964 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:47.273342 (+ 1378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:47.274360 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:47.274925 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:47.275023 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:47.277682 (+ 2659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:47:47.277686 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:47:47.277686 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:47:47.277699 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:47:47.280159 (+ 2460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:47.280565 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:47.280568 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:47.280569 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:47.280690 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:47.280910 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:47.280911 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:47.280911 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:47.281225 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:47:47.281747 (+ 522us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:47.281817 (+ 70us) server_negotiation.cc:299] Negotiation successful
0807 08:47:47.281948 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":411,"thread_start_us":371,"threads_started":1}
I20260807 08:47:47.287103 8354 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:47:47.287603 8354 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:47:47.294296 8354 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 935be513e068495c9d751b758fba8f87: Generated new TSK 0
I20260807 08:47:47.294461 8354 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.159.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.159.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.159.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.129.keytab.
Entry for principal kudu/127.4.159.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.129.keytab.
I20260807 08:47:47.322120 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786092453486433-4734-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786092453486433-4734-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786092453486433-4734-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786092453486433-4734-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.159.129:0
--local_ip_for_outbound_sockets=127.4.159.129
--webserver_interface=127.4.159.129
--webserver_port=0
--tserver_master_addrs=127.4.159.190:43537
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.129.keytab
--principal=kudu/127.4.159.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:36675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:47:47.400215 8370 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:47:47.400394 8370 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:47:47.400415 8370 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:47:47.401530 8370 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:47:47.401571 8370 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:47:47.401597 8370 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.159.129
I20260807 08:47:47.403014 8370 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:36675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786092453486433-4734-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786092453486433-4734-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.129.keytab
--principal=kudu/127.4.159.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.159.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786092453486433-4734-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.159.129
--webserver_port=0
--tserver_master_addrs=127.4.159.190:43537
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8370
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.159.129
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786092453486433-4734-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:47:47.403215 8370 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:47:47.403422 8370 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:47:47.407881 8377 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:47:47.407884 8381 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:47:47.408025 8370 server_base.cc:1034] running on GCE node
W20260807 08:47:47.407982 8379 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:47:47.408330 8370 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:47:47.408551 8370 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:47:47.409686 8370 hybrid_clock.cc:648] HybridClock initialized: now 1786092467409670 us; error 32 us; skew 500 ppm
Aug 07 08:47:47 dist-test-slave-6zbq krb5kdc[8218](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092467, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:47:47.412001 8370 init.cc:377] Logged in from keytab as kudu/127.4.159.129@KRBTEST.COM (short username kudu)
I20260807 08:47:47.412377 8370 webserver.cc:460] Webserver started at http://127.4.159.129:39877/ using document root <none> and password file <none>
I20260807 08:47:47.412557 8370 fs_manager.cc:359] Metadata directory not provided
I20260807 08:47:47.412603 8370 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:47:47.412725 8370 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:47:47.413651 8370 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786092453486433-4734-0/minicluster-data/ts-0/data/instance:
uuid: "f927e49cc3f84711961ccb630e9b5163"
format_stamp: "Formatted at 2026-08-07 08:47:47 on dist-test-slave-6zbq"
server_key: "be1d90b413dcbcae0f546e9901004af2"
server_key_iv: "17acc8fe07d29d963ee4ad9404f87993"
server_key_version: "encryptionkey@0"
I20260807 08:47:47.413970 8370 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786092453486433-4734-0/minicluster-data/ts-0/wal/instance:
uuid: "f927e49cc3f84711961ccb630e9b5163"
format_stamp: "Formatted at 2026-08-07 08:47:47 on dist-test-slave-6zbq"
server_key: "be1d90b413dcbcae0f546e9901004af2"
server_key_iv: "17acc8fe07d29d963ee4ad9404f87993"
server_key_version: "encryptionkey@0"
I20260807 08:47:47.415221 8370 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 08:47:47.415843 8392 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:47.415997 8370 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 08:47:47.416072 8370 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786092453486433-4734-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786092453486433-4734-0/minicluster-data/ts-0/wal
uuid: "f927e49cc3f84711961ccb630e9b5163"
format_stamp: "Formatted at 2026-08-07 08:47:47 on dist-test-slave-6zbq"
server_key: "be1d90b413dcbcae0f546e9901004af2"
server_key_iv: "17acc8fe07d29d963ee4ad9404f87993"
server_key_version: "encryptionkey@0"
I20260807 08:47:47.416138 8370 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786092453486433-4734-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786092453486433-4734-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786092453486433-4734-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:47:47.431778 8370 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:47:47.433431 8370 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:47:47.433544 8370 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:47:47.433729 8370 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:47:47.433992 8370 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:47:47.434023 8370 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:47.434042 8370 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:47:47.434065 8370 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:47.439406 8370 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.129:34363
I20260807 08:47:47.439443 8555 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.129:34363 every 8 connection(s)
I20260807 08:47:47.439697 8370 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786092453486433-4734-0/minicluster-data/ts-0/data/info.pb
Aug 07 08:47:47 dist-test-slave-6zbq krb5kdc[8218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092467, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.129@KRBTEST.COM for kudu/127.4.159.190@KRBTEST.COM
I20260807 08:47:47.446311 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 8370
I20260807 08:47:47.446426 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786092453486433-4734-0/minicluster-data/ts-0/wal/instance
I20260807 08:47:47.446586 4734 external_mini_cluster.cc:1428] Setting key 9437ba9e39f69684257e44b32b2a60d8
I20260807 08:47:47.447819 8341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:47.440274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.129:45905 (local address 127.4.159.190:43537)
0807 08:47:47.440325 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:47.440327 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:47.440497 (+ 170us) server_negotiation.cc:407] Connection header received
0807 08:47:47.440937 (+ 440us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:47.440938 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:47.440951 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:47.441002 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:47:47.441881 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:47.442404 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:47.443325 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:47.443520 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:47.443574 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:47.445371 (+ 1797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:47:47.445372 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:47:47.445372 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:47:47.445380 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:47:47.446486 (+ 1106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:47.446890 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:47.446891 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:47.446892 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:47.446958 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:47.447185 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:47.447186 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:47.447186 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:47.447329 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:47:47.447392 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:47.447643 (+ 251us) server_negotiation.cc:299] Negotiation successful
0807 08:47:47.447731 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260807 08:47:47.447956 8559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:47.440236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:43537 (local address 127.4.159.129:45905)
0807 08:47:47.440393 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0807 08:47:47.440431 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 08:47:47.440848 (+ 417us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:47:47.441076 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:47:47.441091 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:47:47.441608 (+ 517us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:47:47.441782 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:47.441786 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:47.442495 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:47.442497 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:47.443245 (+ 748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:47.443248 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:47.443589 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:47.443590 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:47.443697 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:47.444060 (+ 363us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:47:47.444067 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:47:47.445284 (+ 1217us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:47:47.446549 (+ 1265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:47:47.446556 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:47:47.446558 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:47:47.446801 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:47:47.447015 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:47:47.447016 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:47:47.447016 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:47:47.447118 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:47:47.447397 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:47:47.447406 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:47:47.447521 (+ 115us) client_negotiation.cc:769] Sending connection context
0807 08:47:47.447609 (+ 88us) client_negotiation.cc:240] Negotiation successful
0807 08:47:47.447749 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":80,"threads_started":1}
I20260807 08:47:47.448382 8556 heartbeater.cc:347] Connected to a master server at 127.4.159.190:43537
I20260807 08:47:47.448477 8556 heartbeater.cc:464] Registering TS with master...
I20260807 08:47:47.448654 8556 heartbeater.cc:511] Master 127.4.159.190:43537 requested a full tablet report, sending...
I20260807 08:47:47.449165 8272 ts_manager.cc:194] Registered new tserver with Master: f927e49cc3f84711961ccb630e9b5163 (127.4.159.129:34363)
I20260807 08:47:47.449755 8272 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.159.129@KRBTEST.COM'} at 127.4.159.129:45905
WARNING: no policy specified for kudu/127.4.159.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.159.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.159.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.130.keytab.
Entry for principal kudu/127.4.159.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.130.keytab.
I20260807 08:47:47.469655 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786092453486433-4734-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786092453486433-4734-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786092453486433-4734-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786092453486433-4734-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.159.130:0
--local_ip_for_outbound_sockets=127.4.159.130
--webserver_interface=127.4.159.130
--webserver_port=0
--tserver_master_addrs=127.4.159.190:43537
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.130.keytab
--principal=kudu/127.4.159.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:36675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:47:47.546914 8563 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:47:47.547094 8563 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:47:47.547117 8563 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:47:47.548233 8563 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:47:47.548271 8563 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:47:47.548296 8563 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.159.130
I20260807 08:47:47.549659 8563 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:36675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786092453486433-4734-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786092453486433-4734-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.130.keytab
--principal=kudu/127.4.159.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.159.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786092453486433-4734-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.159.130
--webserver_port=0
--tserver_master_addrs=127.4.159.190:43537
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8563
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.159.130
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786092453486433-4734-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:47:47.549865 8563 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:47:47.550048 8563 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:47:47.554551 8571 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:47:47.554558 8573 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:47:47.554612 8563 server_base.cc:1034] running on GCE node
W20260807 08:47:47.554584 8570 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:47:47.554930 8563 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:47:47.555150 8563 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:47:47.556288 8563 hybrid_clock.cc:648] HybridClock initialized: now 1786092467556271 us; error 29 us; skew 500 ppm
Aug 07 08:47:47 dist-test-slave-6zbq krb5kdc[8218](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092467, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:47:47.558552 8563 init.cc:377] Logged in from keytab as kudu/127.4.159.130@KRBTEST.COM (short username kudu)
I20260807 08:47:47.558969 8563 webserver.cc:460] Webserver started at http://127.4.159.130:35283/ using document root <none> and password file <none>
I20260807 08:47:47.559134 8563 fs_manager.cc:359] Metadata directory not provided
I20260807 08:47:47.559170 8563 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:47:47.559255 8563 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:47:47.560153 8563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786092453486433-4734-0/minicluster-data/ts-1/data/instance:
uuid: "8eb8040da1d54166b57731684426df0f"
format_stamp: "Formatted at 2026-08-07 08:47:47 on dist-test-slave-6zbq"
server_key: "3a862319a584c6051d91c9b2077ec65e"
server_key_iv: "4f4031450199574da0759144553c0869"
server_key_version: "encryptionkey@0"
I20260807 08:47:47.560436 8563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786092453486433-4734-0/minicluster-data/ts-1/wal/instance:
uuid: "8eb8040da1d54166b57731684426df0f"
format_stamp: "Formatted at 2026-08-07 08:47:47 on dist-test-slave-6zbq"
server_key: "3a862319a584c6051d91c9b2077ec65e"
server_key_iv: "4f4031450199574da0759144553c0869"
server_key_version: "encryptionkey@0"
I20260807 08:47:47.561607 8563 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 08:47:47.562240 8583 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:47.562418 8563 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 08:47:47.562505 8563 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786092453486433-4734-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786092453486433-4734-0/minicluster-data/ts-1/wal
uuid: "8eb8040da1d54166b57731684426df0f"
format_stamp: "Formatted at 2026-08-07 08:47:47 on dist-test-slave-6zbq"
server_key: "3a862319a584c6051d91c9b2077ec65e"
server_key_iv: "4f4031450199574da0759144553c0869"
server_key_version: "encryptionkey@0"
I20260807 08:47:47.562566 8563 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786092453486433-4734-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786092453486433-4734-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786092453486433-4734-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:47:47.570684 8563 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:47:47.572281 8563 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:47:47.572400 8563 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:47:47.572593 8563 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:47:47.572849 8563 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:47:47.572881 8563 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:47.572906 8563 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:47:47.572942 8563 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:47.578042 8563 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.130:37077
I20260807 08:47:47.578117 8744 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.130:37077 every 8 connection(s)
I20260807 08:47:47.578358 8563 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786092453486433-4734-0/minicluster-data/ts-1/data/info.pb
I20260807 08:47:47.583431 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 8563
I20260807 08:47:47.583525 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786092453486433-4734-0/minicluster-data/ts-1/wal/instance
I20260807 08:47:47.583714 4734 external_mini_cluster.cc:1428] Setting key 10ac09338faeec2f37bbe3982d54ec74
Aug 07 08:47:47 dist-test-slave-6zbq krb5kdc[8218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092467, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.130@KRBTEST.COM for kudu/127.4.159.190@KRBTEST.COM
I20260807 08:47:47.590819 8341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:47.578968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.130:56181 (local address 127.4.159.190:43537)
0807 08:47:47.579035 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:47.579037 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:47.579190 (+ 153us) server_negotiation.cc:407] Connection header received
0807 08:47:47.579663 (+ 473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:47.579665 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:47.579679 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:47.579733 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:47:47.581525 (+ 1792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:47.582087 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:47.583060 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:47.583261 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:47.583322 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:47.586110 (+ 2788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:47:47.586111 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:47:47.586111 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:47:47.586120 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:47:47.589357 (+ 3237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:47.589813 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:47.589815 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:47.589815 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:47.589878 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:47.590148 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:47.590149 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:47.590149 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:47.590262 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:47:47.590319 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:47.590650 (+ 331us) server_negotiation.cc:299] Negotiation successful
0807 08:47:47.590733 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260807 08:47:47.591270 8752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:47.578913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:43537 (local address 127.4.159.130:56181)
0807 08:47:47.579086 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0807 08:47:47.579122 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 08:47:47.579523 (+ 401us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:47:47.580511 (+ 988us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:47:47.580515 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:47:47.581186 (+ 671us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:47:47.581420 (+ 234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:47.581425 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:47.582173 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:47.582174 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:47.582977 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:47.582979 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:47.583354 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:47.583356 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:47.583512 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:47.584029 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:47:47.584039 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:47:47.585956 (+ 1917us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:47:47.589426 (+ 3470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:47:47.589451 (+ 25us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:47:47.589453 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:47:47.589730 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:47:47.589939 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:47:47.589940 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:47:47.589940 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:47:47.590084 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:47:47.590319 (+ 235us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:47:47.590326 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:47:47.590475 (+ 149us) client_negotiation.cc:769] Sending connection context
0807 08:47:47.590906 (+ 431us) client_negotiation.cc:240] Negotiation successful
0807 08:47:47.591073 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":83,"threads_started":1}
I20260807 08:47:47.591678 8745 heartbeater.cc:347] Connected to a master server at 127.4.159.190:43537
I20260807 08:47:47.591775 8745 heartbeater.cc:464] Registering TS with master...
I20260807 08:47:47.591952 8745 heartbeater.cc:511] Master 127.4.159.190:43537 requested a full tablet report, sending...
I20260807 08:47:47.592460 8272 ts_manager.cc:194] Registered new tserver with Master: 8eb8040da1d54166b57731684426df0f (127.4.159.130:37077)
I20260807 08:47:47.592904 8272 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.159.130@KRBTEST.COM'} at 127.4.159.130:56181
WARNING: no policy specified for kudu/127.4.159.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.159.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.159.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.131.keytab.
Entry for principal kudu/127.4.159.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.131.keytab.
I20260807 08:47:47.614241 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786092453486433-4734-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786092453486433-4734-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786092453486433-4734-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786092453486433-4734-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.159.131:0
--local_ip_for_outbound_sockets=127.4.159.131
--webserver_interface=127.4.159.131
--webserver_port=0
--tserver_master_addrs=127.4.159.190:43537
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.131.keytab
--principal=kudu/127.4.159.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:36675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:47:47.695526 8756 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:47:47.695704 8756 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:47:47.695735 8756 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:47:47.696841 8756 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:47:47.696878 8756 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:47:47.696913 8756 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.159.131
I20260807 08:47:47.698276 8756 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:36675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786092453486433-4734-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786092453486433-4734-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.131.keytab
--principal=kudu/127.4.159.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.159.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786092453486433-4734-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.159.131
--webserver_port=0
--tserver_master_addrs=127.4.159.190:43537
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8756
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.159.131
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786092453486433-4734-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:47:47.698504 8756 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:47:47.698699 8756 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:47:47.703112 8763 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:47:47.703115 8766 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:47:47.703109 8762 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:47:47.703191 8756 server_base.cc:1034] running on GCE node
I20260807 08:47:47.703493 8756 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:47:47.703678 8756 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:47:47.704805 8756 hybrid_clock.cc:648] HybridClock initialized: now 1786092467704794 us; error 28 us; skew 500 ppm
Aug 07 08:47:47 dist-test-slave-6zbq krb5kdc[8218](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092467, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:47:47.707039 8756 init.cc:377] Logged in from keytab as kudu/127.4.159.131@KRBTEST.COM (short username kudu)
I20260807 08:47:47.707433 8756 webserver.cc:460] Webserver started at http://127.4.159.131:37813/ using document root <none> and password file <none>
I20260807 08:47:47.707613 8756 fs_manager.cc:359] Metadata directory not provided
I20260807 08:47:47.707654 8756 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:47:47.707765 8756 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:47:47.708714 8756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786092453486433-4734-0/minicluster-data/ts-2/data/instance:
uuid: "098a73f691fb4354befe2623822bfeb9"
format_stamp: "Formatted at 2026-08-07 08:47:47 on dist-test-slave-6zbq"
server_key: "3c96fcd04bc7988cd97de514e290c2fc"
server_key_iv: "e7df1c09046a6fc69c922472a0ffd9c6"
server_key_version: "encryptionkey@0"
I20260807 08:47:47.709014 8756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786092453486433-4734-0/minicluster-data/ts-2/wal/instance:
uuid: "098a73f691fb4354befe2623822bfeb9"
format_stamp: "Formatted at 2026-08-07 08:47:47 on dist-test-slave-6zbq"
server_key: "3c96fcd04bc7988cd97de514e290c2fc"
server_key_iv: "e7df1c09046a6fc69c922472a0ffd9c6"
server_key_version: "encryptionkey@0"
I20260807 08:47:47.710181 8756 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 08:47:47.710873 8777 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:47.711016 8756 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:47.711099 8756 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786092453486433-4734-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786092453486433-4734-0/minicluster-data/ts-2/wal
uuid: "098a73f691fb4354befe2623822bfeb9"
format_stamp: "Formatted at 2026-08-07 08:47:47 on dist-test-slave-6zbq"
server_key: "3c96fcd04bc7988cd97de514e290c2fc"
server_key_iv: "e7df1c09046a6fc69c922472a0ffd9c6"
server_key_version: "encryptionkey@0"
I20260807 08:47:47.711163 8756 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786092453486433-4734-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786092453486433-4734-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786092453486433-4734-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:47:47.725665 8756 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:47:47.727352 8756 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:47:47.727468 8756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:47:47.727656 8756 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:47:47.727919 8756 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:47:47.727952 8756 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:47.727972 8756 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:47:47.727995 8756 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:47.733429 8756 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.131:45515
I20260807 08:47:47.733485 8930 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.131:45515 every 8 connection(s)
I20260807 08:47:47.733747 8756 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786092453486433-4734-0/minicluster-data/ts-2/data/info.pb
Aug 07 08:47:47 dist-test-slave-6zbq krb5kdc[8218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092467, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.131@KRBTEST.COM for kudu/127.4.159.190@KRBTEST.COM
I20260807 08:47:47.738972 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 8756
I20260807 08:47:47.739063 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786092453486433-4734-0/minicluster-data/ts-2/wal/instance
I20260807 08:47:47.739261 4734 external_mini_cluster.cc:1428] Setting key 16bcd6fa61edb2a6f357cf3ec8bae8d6
I20260807 08:47:47.742096 8341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:47.734400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.131:60087 (local address 127.4.159.190:43537)
0807 08:47:47.734466 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:47.734468 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:47.734621 (+ 153us) server_negotiation.cc:407] Connection header received
0807 08:47:47.735046 (+ 425us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:47.735047 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:47.735057 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:47.735115 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:47:47.735917 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:47.736403 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:47.737264 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:47.737434 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:47.737476 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:47.739516 (+ 2040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:47:47.739518 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:47:47.739518 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:47:47.739527 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:47:47.740857 (+ 1330us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:47.741224 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:47.741225 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:47.741226 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:47.741284 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:47.741501 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:47.741502 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:47.741502 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:47.741613 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:47:47.741667 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:47.741932 (+ 265us) server_negotiation.cc:299] Negotiation successful
0807 08:47:47.742010 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260807 08:47:47.742268 8933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:47.734362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:43537 (local address 127.4.159.131:60087)
0807 08:47:47.734513 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0807 08:47:47.734545 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0807 08:47:47.734961 (+ 416us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:47:47.735162 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:47:47.735165 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:47:47.735672 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:47:47.735851 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:47.735856 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:47.736463 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:47.736463 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:47.737204 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:47.737206 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:47.737485 (+ 279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:47.737485 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:47.737574 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:47.737993 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:47:47.738002 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:47:47.739414 (+ 1412us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:47:47.740932 (+ 1518us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:47:47.740939 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:47:47.740941 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:47:47.741159 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:47:47.741342 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:47:47.741343 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:47:47.741343 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:47:47.741446 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:47:47.741669 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:47:47.741676 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:47:47.741798 (+ 122us) client_negotiation.cc:769] Sending connection context
0807 08:47:47.741891 (+ 93us) client_negotiation.cc:240] Negotiation successful
0807 08:47:47.742031 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":77,"threads_started":1}
I20260807 08:47:47.742702 8931 heartbeater.cc:347] Connected to a master server at 127.4.159.190:43537
I20260807 08:47:47.742780 8931 heartbeater.cc:464] Registering TS with master...
I20260807 08:47:47.742935 8931 heartbeater.cc:511] Master 127.4.159.190:43537 requested a full tablet report, sending...
I20260807 08:47:47.743263 8272 ts_manager.cc:194] Registered new tserver with Master: 098a73f691fb4354befe2623822bfeb9 (127.4.159.131:45515)
I20260807 08:47:47.743595 8272 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.159.131@KRBTEST.COM'} at 127.4.159.131:60087
I20260807 08:47:47.750801 4734 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260807 08:47:47.761865 8341 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:47:47.761062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47546 (local address 127.4.159.190:43537)
0807 08:47:47.761130 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:47.761132 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:47.761230 (+ 98us) server_negotiation.cc:407] Connection header received
0807 08:47:47.761292 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:47.761292 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:47.761301 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:47.761347 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:47:47.761809 (+ 462us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47546: BlockingRecv error: recv got EOF from 127.0.0.1:47546 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
I20260807 08:47:47.762722 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 8370
I20260807 08:47:47.783859 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 8563
I20260807 08:47:47.792725 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 8756
I20260807 08:47:47.814536 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 8232
2026-08-07T08:47:47Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (1881 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786092453486433-4734-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:47:47 dist-test-slave-6zbq krb5kdc[8950](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:47:47 dist-test-slave-6zbq krb5kdc[8950](info): set up 2 sockets
Aug 07 08:47:47 dist-test-slave-6zbq krb5kdc[8950](info): commencing operation
krb5kdc: starting...
W20260807 08:47:48.932139 4734 mini_kdc.cc:121] Time spent starting KDC: real 1.095s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786092453486433-4734-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786092453486433-4734-0/krb5kdc/test-user.keytab.
Aug 07 08:47:49 dist-test-slave-6zbq krb5kdc[8950](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092469, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:47:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:47:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.159.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.159.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.159.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab.
Entry for principal kudu/127.4.159.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab.
I20260807 08:47:49.036725 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786092453486433-4734-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786092453486433-4734-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786092453486433-4734-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.159.190:42271
--webserver_interface=127.4.159.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab
--principal=kudu/127.4.159.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:42885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.159.190:42271
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:47:49.114357 8971 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:47:49.114526 8971 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:47:49.114545 8971 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:47:49.115810 8971 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:47:49.115861 8971 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:47:49.115881 8971 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:47:49.115899 8971 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:47:49.115973 8971 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:47:49.117465 8971 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:42885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786092453486433-4734-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786092453486433-4734-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.159.190:42271
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab
--principal=kudu/127.4.159.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.159.190:42271
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.159.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8971
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786092453486433-4734-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:47:49.117678 8971 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:47:49.117877 8971 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:47:49.122219 8979 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:47:49.122273 8983 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:47:49.122422 8971 server_base.cc:1034] running on GCE node
W20260807 08:47:49.122467 8981 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:47:49.122681 8971 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:47:49.122939 8971 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:47:49.124064 8971 hybrid_clock.cc:648] HybridClock initialized: now 1786092469124056 us; error 26 us; skew 500 ppm
Aug 07 08:47:49 dist-test-slave-6zbq krb5kdc[8950](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092469, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:47:49.130071 8971 init.cc:377] Logged in from keytab as kudu/127.4.159.190@KRBTEST.COM (short username kudu)
I20260807 08:47:49.130512 8971 webserver.cc:460] Webserver started at http://127.4.159.190:41465/ using document root <none> and password file <none>
I20260807 08:47:49.130756 8971 fs_manager.cc:359] Metadata directory not provided
I20260807 08:47:49.130817 8971 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:47:49.130946 8971 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:47:49.132032 8971 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786092453486433-4734-0/minicluster-data/master-0/data/instance:
uuid: "aac4ec658f8f4b29b604764aff2e97c2"
format_stamp: "Formatted at 2026-08-07 08:47:49 on dist-test-slave-6zbq"
server_key: "eeb4fb5ebaa71164e8a6f0cc46d704eb"
server_key_iv: "c29234bbbdd9583e07ee06edbeb1c3b7"
server_key_version: "encryptionkey@0"
I20260807 08:47:49.132321 8971 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786092453486433-4734-0/minicluster-data/master-0/wal/instance:
uuid: "aac4ec658f8f4b29b604764aff2e97c2"
format_stamp: "Formatted at 2026-08-07 08:47:49 on dist-test-slave-6zbq"
server_key: "eeb4fb5ebaa71164e8a6f0cc46d704eb"
server_key_iv: "c29234bbbdd9583e07ee06edbeb1c3b7"
server_key_version: "encryptionkey@0"
I20260807 08:47:49.133404 8971 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 08:47:49.133991 8991 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:49.134141 8971 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 08:47:49.134228 8971 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786092453486433-4734-0/minicluster-data/master-0/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786092453486433-4734-0/minicluster-data/master-0/wal
uuid: "aac4ec658f8f4b29b604764aff2e97c2"
format_stamp: "Formatted at 2026-08-07 08:47:49 on dist-test-slave-6zbq"
server_key: "eeb4fb5ebaa71164e8a6f0cc46d704eb"
server_key_iv: "c29234bbbdd9583e07ee06edbeb1c3b7"
server_key_version: "encryptionkey@0"
I20260807 08:47:49.134294 8971 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786092453486433-4734-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786092453486433-4734-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786092453486433-4734-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:47:49.144749 8971 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:47:49.146648 8971 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:47:49.146773 8971 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:47:49.150439 8971 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.190:42271
I20260807 08:47:49.150489 9069 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.190:42271 every 8 connection(s)
I20260807 08:47:49.150763 8971 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
I20260807 08:47:49.151489 9070 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:47:49.153501 9070 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aac4ec658f8f4b29b604764aff2e97c2: Bootstrap starting.
I20260807 08:47:49.154047 9070 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aac4ec658f8f4b29b604764aff2e97c2: Neither blocks nor log segments found. Creating new log.
I20260807 08:47:49.154323 9070 log.cc:824] T 00000000000000000000000000000000 P aac4ec658f8f4b29b604764aff2e97c2: Log is configured to *not* fsync() on all Append() calls
I20260807 08:47:49.154942 9070 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aac4ec658f8f4b29b604764aff2e97c2: No bootstrap required, opened a new log
I20260807 08:47:49.156441 9070 raft_consensus.cc:348] T 00000000000000000000000000000000 P aac4ec658f8f4b29b604764aff2e97c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aac4ec658f8f4b29b604764aff2e97c2" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 42271 } }
I20260807 08:47:49.156565 9070 raft_consensus.cc:374] T 00000000000000000000000000000000 P aac4ec658f8f4b29b604764aff2e97c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:47:49.156598 9070 raft_consensus.cc:729] T 00000000000000000000000000000000 P aac4ec658f8f4b29b604764aff2e97c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aac4ec658f8f4b29b604764aff2e97c2, State: Initialized, Role: FOLLOWER
I20260807 08:47:49.156729 9070 consensus_queue.cc:260] T 00000000000000000000000000000000 P aac4ec658f8f4b29b604764aff2e97c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aac4ec658f8f4b29b604764aff2e97c2" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 42271 } }
I20260807 08:47:49.156785 9070 raft_consensus.cc:388] T 00000000000000000000000000000000 P aac4ec658f8f4b29b604764aff2e97c2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:47:49.156816 9070 raft_consensus.cc:482] T 00000000000000000000000000000000 P aac4ec658f8f4b29b604764aff2e97c2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:47:49.156859 9070 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aac4ec658f8f4b29b604764aff2e97c2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:47:49.157572 9070 raft_consensus.cc:504] T 00000000000000000000000000000000 P aac4ec658f8f4b29b604764aff2e97c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aac4ec658f8f4b29b604764aff2e97c2" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 42271 } }
I20260807 08:47:49.157665 9070 leader_election.cc:302] T 00000000000000000000000000000000 P aac4ec658f8f4b29b604764aff2e97c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aac4ec658f8f4b29b604764aff2e97c2; no voters:
I20260807 08:47:49.157826 9070 leader_election.cc:288] T 00000000000000000000000000000000 P aac4ec658f8f4b29b604764aff2e97c2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:47:49.157864 9075 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aac4ec658f8f4b29b604764aff2e97c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:47:49.158012 9075 raft_consensus.cc:686] T 00000000000000000000000000000000 P aac4ec658f8f4b29b604764aff2e97c2 [term 1 LEADER]: Becoming Leader. State: Replica: aac4ec658f8f4b29b604764aff2e97c2, State: Running, Role: LEADER
I20260807 08:47:49.158048 9070 sys_catalog.cc:564] T 00000000000000000000000000000000 P aac4ec658f8f4b29b604764aff2e97c2 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:47:49.158098 9075 consensus_queue.cc:237] T 00000000000000000000000000000000 P aac4ec658f8f4b29b604764aff2e97c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aac4ec658f8f4b29b604764aff2e97c2" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 42271 } }
I20260807 08:47:49.158461 9077 sys_catalog.cc:455] T 00000000000000000000000000000000 P aac4ec658f8f4b29b604764aff2e97c2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aac4ec658f8f4b29b604764aff2e97c2. Latest consensus state: current_term: 1 leader_uuid: "aac4ec658f8f4b29b604764aff2e97c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aac4ec658f8f4b29b604764aff2e97c2" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 42271 } } }
I20260807 08:47:49.158490 9076 sys_catalog.cc:455] T 00000000000000000000000000000000 P aac4ec658f8f4b29b604764aff2e97c2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "aac4ec658f8f4b29b604764aff2e97c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aac4ec658f8f4b29b604764aff2e97c2" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 42271 } } }
I20260807 08:47:49.158566 9076 sys_catalog.cc:458] T 00000000000000000000000000000000 P aac4ec658f8f4b29b604764aff2e97c2 [sys.catalog]: This master's current role is: LEADER
I20260807 08:47:49.158563 9077 sys_catalog.cc:458] T 00000000000000000000000000000000 P aac4ec658f8f4b29b604764aff2e97c2 [sys.catalog]: This master's current role is: LEADER
I20260807 08:47:49.158842 9085 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:47:49.159307 9085 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:47:49.160518 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 8971
I20260807 08:47:49.160609 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786092453486433-4734-0/minicluster-data/master-0/wal/instance
I20260807 08:47:49.160768 9085 catalog_manager.cc:1347] Generated new cluster ID: 4d4aace286d14207887e716442ad2e59
I20260807 08:47:49.160780 4734 external_mini_cluster.cc:1428] Setting key c49ed174908d3b4ec28cdae66cfd2ec1
I20260807 08:47:49.160820 9085 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 08:47:49 dist-test-slave-6zbq krb5kdc[8950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092469, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.159.190@KRBTEST.COM
I20260807 08:47:49.169301 9102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:49.161154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35526 (local address 127.4.159.190:42271)
0807 08:47:49.161305 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:49.161314 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:49.161398 (+ 84us) server_negotiation.cc:407] Connection header received
0807 08:47:49.161541 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:49.161547 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:49.161588 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:49.161696 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:47:49.162297 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:49.163229 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:49.163942 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:49.164368 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:49.164460 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:49.166562 (+ 2102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:47:49.166566 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:47:49.166566 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:47:49.166576 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:47:49.167958 (+ 1382us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:49.168278 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:49.168280 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:49.168281 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:49.168361 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:49.168533 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:49.168534 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:49.168534 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:49.168747 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:47:49.168891 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:49.168983 (+ 92us) server_negotiation.cc:299] Negotiation successful
0807 08:47:49.169109 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":74,"threads_started":1}
I20260807 08:47:49.170107 4734 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 08:47:49.182680 9085 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:47:49.183169 9102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:49.179981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35532 (local address 127.4.159.190:42271)
0807 08:47:49.180027 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:49.180028 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:49.180145 (+ 117us) server_negotiation.cc:407] Connection header received
0807 08:47:49.180198 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:49.180199 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:49.180216 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:49.180259 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:47:49.180599 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:49.181039 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:49.181710 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:49.181902 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:49.181956 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:49.182738 (+ 782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:47:49.182740 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:47:49.182740 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:47:49.182932 (+ 192us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:47:49.182946 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:47:49.182952 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:47:49.182978 (+ 26us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:49.183077 (+ 99us) server_negotiation.cc:299] Negotiation successful
0807 08:47:49.183101 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260807 08:47:49.183350 9085 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:47:49.193205 9085 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aac4ec658f8f4b29b604764aff2e97c2: Generated new TSK 0
I20260807 08:47:49.193360 9085 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:47:49.197539 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 8971
2026-08-07T08:47:49Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1386 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786092453486433-4734-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:47:49 dist-test-slave-6zbq krb5kdc[9114](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:47:49 dist-test-slave-6zbq krb5kdc[9114](info): set up 2 sockets
Aug 07 08:47:49 dist-test-slave-6zbq krb5kdc[9114](info): commencing operation
krb5kdc: starting...
W20260807 08:47:50.311739 4734 mini_kdc.cc:121] Time spent starting KDC: real 1.088s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786092453486433-4734-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786092453486433-4734-0/krb5kdc/test-user.keytab.
Aug 07 08:47:50 dist-test-slave-6zbq krb5kdc[9114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092470, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:47:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:47:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.159.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.159.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.159.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab.
Entry for principal kudu/127.4.159.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab.
I20260807 08:47:50.419221 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786092453486433-4734-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786092453486433-4734-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786092453486433-4734-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.159.190:37441
--webserver_interface=127.4.159.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab
--principal=kudu/127.4.159.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:42093
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.159.190:37441
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:47:50.499068 9134 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:47:50.499256 9134 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:47:50.499277 9134 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:47:50.500357 9134 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:47:50.500396 9134 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:47:50.500408 9134 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:47:50.500419 9134 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:47:50.500432 9134 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:47:50.501845 9134 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:42093
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786092453486433-4734-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786092453486433-4734-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.159.190:37441
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab
--principal=kudu/127.4.159.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.159.190:37441
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.159.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9134
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786092453486433-4734-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:47:50.502048 9134 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:47:50.502247 9134 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:47:50.506693 9143 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:47:50.506824 9134 server_base.cc:1034] running on GCE node
W20260807 08:47:50.506709 9141 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:47:50.506701 9140 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:47:50.507112 9134 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:47:50.507385 9134 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:47:50.508520 9134 hybrid_clock.cc:648] HybridClock initialized: now 1786092470508507 us; error 29 us; skew 500 ppm
Aug 07 08:47:50 dist-test-slave-6zbq krb5kdc[9114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092470, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:47:50.510776 9134 init.cc:377] Logged in from keytab as kudu/127.4.159.190@KRBTEST.COM (short username kudu)
I20260807 08:47:50.511174 9134 webserver.cc:460] Webserver started at http://127.4.159.190:35899/ using document root <none> and password file <none>
I20260807 08:47:50.511360 9134 fs_manager.cc:359] Metadata directory not provided
I20260807 08:47:50.511405 9134 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:47:50.511538 9134 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:47:50.512471 9134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786092453486433-4734-0/minicluster-data/master-0/data/instance:
uuid: "57be8127015f47ca86618ea4b771c390"
format_stamp: "Formatted at 2026-08-07 08:47:50 on dist-test-slave-6zbq"
server_key: "19e3cc25ea5175eb0f6e0a69b9e64a23"
server_key_iv: "3aee8892ceea17e1e8e394a69776d03e"
server_key_version: "encryptionkey@0"
I20260807 08:47:50.512806 9134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786092453486433-4734-0/minicluster-data/master-0/wal/instance:
uuid: "57be8127015f47ca86618ea4b771c390"
format_stamp: "Formatted at 2026-08-07 08:47:50 on dist-test-slave-6zbq"
server_key: "19e3cc25ea5175eb0f6e0a69b9e64a23"
server_key_iv: "3aee8892ceea17e1e8e394a69776d03e"
server_key_version: "encryptionkey@0"
I20260807 08:47:50.513978 9134 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260807 08:47:50.514638 9152 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:50.514772 9134 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:50.514853 9134 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786092453486433-4734-0/minicluster-data/master-0/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786092453486433-4734-0/minicluster-data/master-0/wal
uuid: "57be8127015f47ca86618ea4b771c390"
format_stamp: "Formatted at 2026-08-07 08:47:50 on dist-test-slave-6zbq"
server_key: "19e3cc25ea5175eb0f6e0a69b9e64a23"
server_key_iv: "3aee8892ceea17e1e8e394a69776d03e"
server_key_version: "encryptionkey@0"
I20260807 08:47:50.514915 9134 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786092453486433-4734-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786092453486433-4734-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786092453486433-4734-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:47:50.526335 9134 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:47:50.527870 9134 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:47:50.527987 9134 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:47:50.531706 9134 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.190:37441
I20260807 08:47:50.531737 9228 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.190:37441 every 8 connection(s)
I20260807 08:47:50.532016 9134 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
I20260807 08:47:50.533034 9230 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:47:50.533546 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 9134
I20260807 08:47:50.533649 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786092453486433-4734-0/minicluster-data/master-0/wal/instance
I20260807 08:47:50.533835 4734 external_mini_cluster.cc:1428] Setting key 33c9e60fc07b5fc12544204393cc6009
I20260807 08:47:50.535979 9230 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57be8127015f47ca86618ea4b771c390: Bootstrap starting.
I20260807 08:47:50.536592 9230 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 57be8127015f47ca86618ea4b771c390: Neither blocks nor log segments found. Creating new log.
I20260807 08:47:50.536875 9230 log.cc:824] T 00000000000000000000000000000000 P 57be8127015f47ca86618ea4b771c390: Log is configured to *not* fsync() on all Append() calls
I20260807 08:47:50.537633 9230 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57be8127015f47ca86618ea4b771c390: No bootstrap required, opened a new log
I20260807 08:47:50.539913 9230 raft_consensus.cc:348] T 00000000000000000000000000000000 P 57be8127015f47ca86618ea4b771c390 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57be8127015f47ca86618ea4b771c390" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 37441 } }
Aug 07 08:47:50 dist-test-slave-6zbq krb5kdc[9114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092470, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.159.190@KRBTEST.COM
I20260807 08:47:50.540038 9230 raft_consensus.cc:374] T 00000000000000000000000000000000 P 57be8127015f47ca86618ea4b771c390 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:47:50.540059 9230 raft_consensus.cc:729] T 00000000000000000000000000000000 P 57be8127015f47ca86618ea4b771c390 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57be8127015f47ca86618ea4b771c390, State: Initialized, Role: FOLLOWER
I20260807 08:47:50.540161 9230 consensus_queue.cc:260] T 00000000000000000000000000000000 P 57be8127015f47ca86618ea4b771c390 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57be8127015f47ca86618ea4b771c390" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 37441 } }
I20260807 08:47:50.540241 9230 raft_consensus.cc:388] T 00000000000000000000000000000000 P 57be8127015f47ca86618ea4b771c390 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:47:50.540262 9230 raft_consensus.cc:482] T 00000000000000000000000000000000 P 57be8127015f47ca86618ea4b771c390 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:47:50.540305 9230 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 57be8127015f47ca86618ea4b771c390 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:47:50.540813 9230 raft_consensus.cc:504] T 00000000000000000000000000000000 P 57be8127015f47ca86618ea4b771c390 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57be8127015f47ca86618ea4b771c390" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 37441 } }
I20260807 08:47:50.540925 9230 leader_election.cc:302] T 00000000000000000000000000000000 P 57be8127015f47ca86618ea4b771c390 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 57be8127015f47ca86618ea4b771c390; no voters:
I20260807 08:47:50.541097 9230 leader_election.cc:288] T 00000000000000000000000000000000 P 57be8127015f47ca86618ea4b771c390 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:47:50.541136 9237 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 57be8127015f47ca86618ea4b771c390 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:47:50.541347 9237 raft_consensus.cc:686] T 00000000000000000000000000000000 P 57be8127015f47ca86618ea4b771c390 [term 1 LEADER]: Becoming Leader. State: Replica: 57be8127015f47ca86618ea4b771c390, State: Running, Role: LEADER
I20260807 08:47:50.541373 9230 sys_catalog.cc:564] T 00000000000000000000000000000000 P 57be8127015f47ca86618ea4b771c390 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:47:50.541508 9237 consensus_queue.cc:237] T 00000000000000000000000000000000 P 57be8127015f47ca86618ea4b771c390 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57be8127015f47ca86618ea4b771c390" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 37441 } }
I20260807 08:47:50.541826 9237 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57be8127015f47ca86618ea4b771c390 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 57be8127015f47ca86618ea4b771c390. Latest consensus state: current_term: 1 leader_uuid: "57be8127015f47ca86618ea4b771c390" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57be8127015f47ca86618ea4b771c390" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 37441 } } }
I20260807 08:47:50.541946 9237 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57be8127015f47ca86618ea4b771c390 [sys.catalog]: This master's current role is: LEADER
I20260807 08:47:50.542232 9246 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:47:50.542800 9238 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57be8127015f47ca86618ea4b771c390 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "57be8127015f47ca86618ea4b771c390" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57be8127015f47ca86618ea4b771c390" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 37441 } } }
I20260807 08:47:50.542915 9238 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57be8127015f47ca86618ea4b771c390 [sys.catalog]: This master's current role is: LEADER
I20260807 08:47:50.542930 9246 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:47:50.544107 9234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:50.534375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32804 (local address 127.4.159.190:37441)
0807 08:47:50.534512 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:50.534520 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:50.534548 (+ 28us) server_negotiation.cc:407] Connection header received
0807 08:47:50.534975 (+ 427us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:50.534980 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:50.535023 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:50.535127 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:47:50.535847 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:50.536728 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:50.537596 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:50.538010 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:50.538088 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:50.540551 (+ 2463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:47:50.540555 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:47:50.540555 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:47:50.540565 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:47:50.542572 (+ 2007us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:50.542973 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:50.542976 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:50.542977 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:50.543085 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:50.543297 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:50.543297 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:50.543298 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:50.543518 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:47:50.543667 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:50.543778 (+ 111us) server_negotiation.cc:299] Negotiation successful
0807 08:47:50.543919 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91,"thread_start_us":59,"threads_started":1}
I20260807 08:47:50.544847 9246 catalog_manager.cc:1347] Generated new cluster ID: 81bf5e6a41d4491d927cc8f9d1b627ba
I20260807 08:47:50.544898 9246 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:47:50.561497 9246 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:47:50.561977 9246 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:47:50.564069 9246 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 57be8127015f47ca86618ea4b771c390: Generated new TSK 0
I20260807 08:47:50.564219 9246 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.159.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.159.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.159.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.129.keytab.
Entry for principal kudu/127.4.159.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.129.keytab.
I20260807 08:47:50.592235 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786092453486433-4734-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786092453486433-4734-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786092453486433-4734-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786092453486433-4734-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.159.129:0
--local_ip_for_outbound_sockets=127.4.159.129
--webserver_interface=127.4.159.129
--webserver_port=0
--tserver_master_addrs=127.4.159.190:37441
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.129.keytab
--principal=kudu/127.4.159.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:42093
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:47:50.689730 9266 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:47:50.689913 9266 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:47:50.689934 9266 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:47:50.691126 9266 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:47:50.691167 9266 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:47:50.691192 9266 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.159.129
I20260807 08:47:50.692584 9266 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:42093
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786092453486433-4734-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786092453486433-4734-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.129.keytab
--principal=kudu/127.4.159.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.159.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786092453486433-4734-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.159.129
--webserver_port=0
--tserver_master_addrs=127.4.159.190:37441
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9266
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.159.129
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786092453486433-4734-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:47:50.692813 9266 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:47:50.693008 9266 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:47:50.697520 9278 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:47:50.697618 9266 server_base.cc:1034] running on GCE node
W20260807 08:47:50.697526 9280 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:47:50.697540 9277 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:47:50.697906 9266 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:47:50.698127 9266 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:47:50.699256 9266 hybrid_clock.cc:648] HybridClock initialized: now 1786092470699243 us; error 29 us; skew 500 ppm
Aug 07 08:47:50 dist-test-slave-6zbq krb5kdc[9114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092470, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:47:50.701496 9266 init.cc:377] Logged in from keytab as kudu/127.4.159.129@KRBTEST.COM (short username kudu)
I20260807 08:47:50.701879 9266 webserver.cc:460] Webserver started at http://127.4.159.129:42101/ using document root <none> and password file <none>
I20260807 08:47:50.702042 9266 fs_manager.cc:359] Metadata directory not provided
I20260807 08:47:50.702078 9266 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:47:50.702165 9266 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:47:50.703116 9266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786092453486433-4734-0/minicluster-data/ts-0/data/instance:
uuid: "2263dd590c2e4aeaa0b9aff4afe63bdb"
format_stamp: "Formatted at 2026-08-07 08:47:50 on dist-test-slave-6zbq"
server_key: "96a43a8179785fc82ef25a7958df41e6"
server_key_iv: "23812320cbf3a236d3ecffd79865839a"
server_key_version: "encryptionkey@0"
I20260807 08:47:50.703431 9266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786092453486433-4734-0/minicluster-data/ts-0/wal/instance:
uuid: "2263dd590c2e4aeaa0b9aff4afe63bdb"
format_stamp: "Formatted at 2026-08-07 08:47:50 on dist-test-slave-6zbq"
server_key: "96a43a8179785fc82ef25a7958df41e6"
server_key_iv: "23812320cbf3a236d3ecffd79865839a"
server_key_version: "encryptionkey@0"
I20260807 08:47:50.704567 9266 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260807 08:47:50.705163 9289 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:50.705298 9266 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 08:47:50.705379 9266 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786092453486433-4734-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786092453486433-4734-0/minicluster-data/ts-0/wal
uuid: "2263dd590c2e4aeaa0b9aff4afe63bdb"
format_stamp: "Formatted at 2026-08-07 08:47:50 on dist-test-slave-6zbq"
server_key: "96a43a8179785fc82ef25a7958df41e6"
server_key_iv: "23812320cbf3a236d3ecffd79865839a"
server_key_version: "encryptionkey@0"
I20260807 08:47:50.705436 9266 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786092453486433-4734-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786092453486433-4734-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786092453486433-4734-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:47:50.728246 9266 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:47:50.730021 9266 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:47:50.730146 9266 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:47:50.730432 9266 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:47:50.730737 9266 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:47:50.730772 9266 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:50.730798 9266 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:47:50.730822 9266 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:50.736129 9266 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.129:43381
I20260807 08:47:50.736194 9451 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.129:43381 every 8 connection(s)
I20260807 08:47:50.736449 9266 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786092453486433-4734-0/minicluster-data/ts-0/data/info.pb
I20260807 08:47:50.736865 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 9266
I20260807 08:47:50.736953 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786092453486433-4734-0/minicluster-data/ts-0/wal/instance
I20260807 08:47:50.737128 4734 external_mini_cluster.cc:1428] Setting key bc8e10ab535275e204d8705372f56bcc
Aug 07 08:47:50 dist-test-slave-6zbq krb5kdc[9114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092470, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.129@KRBTEST.COM for kudu/127.4.159.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.4.159.130@KRBTEST.COM; defaulting to no policy
I20260807 08:47:50.746095 9234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:50.737390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.129:55635 (local address 127.4.159.190:37441)
0807 08:47:50.737439 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:50.737441 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:50.737453 (+ 12us) server_negotiation.cc:407] Connection header received
0807 08:47:50.737901 (+ 448us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:50.737902 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:50.737914 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:50.737979 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:47:50.738875 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:50.739542 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:50.740814 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:50.741103 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:50.741177 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:50.743528 (+ 2351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:47:50.743529 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:47:50.743529 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:47:50.743536 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:47:50.744770 (+ 1234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:50.745230 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:50.745230 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:50.745231 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:50.745276 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:50.745514 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:50.745515 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:50.745515 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:50.745616 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:47:50.745660 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:50.745963 (+ 303us) server_negotiation.cc:299] Negotiation successful
0807 08:47:50.746031 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 08:47:50.746300 9456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:50.737067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:37441 (local address 127.4.159.129:55635)
0807 08:47:50.737226 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0807 08:47:50.737265 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 08:47:50.737811 (+ 546us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:47:50.738030 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:47:50.738033 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:47:50.738609 (+ 576us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:47:50.738790 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:50.738795 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:50.739627 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:50.739628 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:50.740734 (+ 1106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:50.740737 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:50.741183 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:50.741184 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:50.741281 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:50.741766 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:47:50.741774 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:47:50.743440 (+ 1666us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:47:50.744861 (+ 1421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:47:50.744869 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:47:50.744871 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:47:50.745152 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:47:50.745332 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:47:50.745332 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:47:50.745333 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:47:50.745460 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:47:50.745670 (+ 210us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:47:50.745676 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:47:50.745831 (+ 155us) client_negotiation.cc:769] Sending connection context
0807 08:47:50.745944 (+ 113us) client_negotiation.cc:240] Negotiation successful
0807 08:47:50.746090 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":79,"threads_started":1}
I20260807 08:47:50.746829 9452 heartbeater.cc:347] Connected to a master server at 127.4.159.190:37441
I20260807 08:47:50.746912 9452 heartbeater.cc:464] Registering TS with master...
I20260807 08:47:50.747072 9452 heartbeater.cc:511] Master 127.4.159.190:37441 requested a full tablet report, sending...
I20260807 08:47:50.747536 9182 ts_manager.cc:194] Registered new tserver with Master: 2263dd590c2e4aeaa0b9aff4afe63bdb (127.4.159.129:43381)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.159.130@KRBTEST.COM" created.
I20260807 08:47:50.748101 9182 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.159.129@KRBTEST.COM'} at 127.4.159.129:55635
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.159.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.130.keytab.
Entry for principal kudu/127.4.159.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.130.keytab.
I20260807 08:47:50.760455 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786092453486433-4734-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786092453486433-4734-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786092453486433-4734-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786092453486433-4734-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.159.130:0
--local_ip_for_outbound_sockets=127.4.159.130
--webserver_interface=127.4.159.130
--webserver_port=0
--tserver_master_addrs=127.4.159.190:37441
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.130.keytab
--principal=kudu/127.4.159.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:42093
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:47:50.839418 9459 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:47:50.839589 9459 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:47:50.839610 9459 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:47:50.840723 9459 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:47:50.840761 9459 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:47:50.840785 9459 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.159.130
I20260807 08:47:50.842207 9459 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:42093
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786092453486433-4734-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786092453486433-4734-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.130.keytab
--principal=kudu/127.4.159.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.159.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786092453486433-4734-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.159.130
--webserver_port=0
--tserver_master_addrs=127.4.159.190:37441
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9459
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.159.130
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786092453486433-4734-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:47:50.842435 9459 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:47:50.842639 9459 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:47:50.847038 9470 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:47:50.847061 9466 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:47:50.847057 9467 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:47:50.847208 9459 server_base.cc:1034] running on GCE node
I20260807 08:47:50.847469 9459 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:47:50.847702 9459 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:47:50.848845 9459 hybrid_clock.cc:648] HybridClock initialized: now 1786092470848835 us; error 30 us; skew 500 ppm
Aug 07 08:47:50 dist-test-slave-6zbq krb5kdc[9114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092470, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:47:50.851090 9459 init.cc:377] Logged in from keytab as kudu/127.4.159.130@KRBTEST.COM (short username kudu)
I20260807 08:47:50.851490 9459 webserver.cc:460] Webserver started at http://127.4.159.130:37089/ using document root <none> and password file <none>
I20260807 08:47:50.851706 9459 fs_manager.cc:359] Metadata directory not provided
I20260807 08:47:50.851753 9459 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:47:50.851872 9459 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:47:50.852806 9459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786092453486433-4734-0/minicluster-data/ts-1/data/instance:
uuid: "d9f1d6afac8f40629b11fca74231963c"
format_stamp: "Formatted at 2026-08-07 08:47:50 on dist-test-slave-6zbq"
server_key: "3dbfe6531f8ce66aba82142676589ef8"
server_key_iv: "b9b3aff4f3f91faa5161d7967a11553b"
server_key_version: "encryptionkey@0"
I20260807 08:47:50.853120 9459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786092453486433-4734-0/minicluster-data/ts-1/wal/instance:
uuid: "d9f1d6afac8f40629b11fca74231963c"
format_stamp: "Formatted at 2026-08-07 08:47:50 on dist-test-slave-6zbq"
server_key: "3dbfe6531f8ce66aba82142676589ef8"
server_key_iv: "b9b3aff4f3f91faa5161d7967a11553b"
server_key_version: "encryptionkey@0"
I20260807 08:47:50.854307 9459 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 08:47:50.854995 9480 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:50.855134 9459 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 08:47:50.855214 9459 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786092453486433-4734-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786092453486433-4734-0/minicluster-data/ts-1/wal
uuid: "d9f1d6afac8f40629b11fca74231963c"
format_stamp: "Formatted at 2026-08-07 08:47:50 on dist-test-slave-6zbq"
server_key: "3dbfe6531f8ce66aba82142676589ef8"
server_key_iv: "b9b3aff4f3f91faa5161d7967a11553b"
server_key_version: "encryptionkey@0"
I20260807 08:47:50.855280 9459 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786092453486433-4734-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786092453486433-4734-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786092453486433-4734-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:47:50.870493 9459 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:47:50.872058 9459 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:47:50.872182 9459 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:47:50.872368 9459 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:47:50.872633 9459 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:47:50.872666 9459 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:50.872692 9459 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:47:50.872715 9459 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:50.877952 9459 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.130:46041
I20260807 08:47:50.877988 9647 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.130:46041 every 8 connection(s)
I20260807 08:47:50.878252 9459 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786092453486433-4734-0/minicluster-data/ts-1/data/info.pb
Aug 07 08:47:50 dist-test-slave-6zbq krb5kdc[9114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092470, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.130@KRBTEST.COM for kudu/127.4.159.190@KRBTEST.COM
I20260807 08:47:50.884682 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 9459
I20260807 08:47:50.884775 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786092453486433-4734-0/minicluster-data/ts-1/wal/instance
I20260807 08:47:50.884951 4734 external_mini_cluster.cc:1428] Setting key 1795cc7935a6cc4090a83e0c5c72b4d2
I20260807 08:47:50.886366 9234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:50.878899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.130:40843 (local address 127.4.159.190:37441)
0807 08:47:50.878943 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:50.878945 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:50.879129 (+ 184us) server_negotiation.cc:407] Connection header received
0807 08:47:50.879521 (+ 392us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:50.879522 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:50.879531 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:50.879582 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:47:50.880378 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:50.880852 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:50.881733 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:50.881906 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:50.881949 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:50.883764 (+ 1815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:47:50.883765 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:47:50.883765 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:47:50.883772 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:47:50.884989 (+ 1217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:50.885420 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:50.885421 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:50.885422 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:50.885484 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:50.885741 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:50.885741 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:50.885742 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:50.885841 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:47:50.885890 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:50.886202 (+ 312us) server_negotiation.cc:299] Negotiation successful
0807 08:47:50.886275 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 08:47:50.886526 9651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:50.878865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:37441 (local address 127.4.159.130:40843)
0807 08:47:50.879006 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0807 08:47:50.879043 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 08:47:50.879452 (+ 409us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:47:50.879635 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:47:50.879638 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:47:50.880142 (+ 504us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:47:50.880313 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:50.880317 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:50.880911 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:50.880912 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:50.881674 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:50.881676 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:50.881957 (+ 281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:50.881958 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:50.882047 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:50.882439 (+ 392us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:47:50.882447 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:47:50.883680 (+ 1233us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:47:50.885058 (+ 1378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:47:50.885066 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:47:50.885067 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:47:50.885348 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:47:50.885543 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:47:50.885543 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:47:50.885544 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:47:50.885677 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:47:50.885895 (+ 218us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:47:50.885901 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:47:50.886038 (+ 137us) client_negotiation.cc:769] Sending connection context
0807 08:47:50.886154 (+ 116us) client_negotiation.cc:240] Negotiation successful
0807 08:47:50.886303 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":64,"threads_started":1}
I20260807 08:47:50.886950 9649 heartbeater.cc:347] Connected to a master server at 127.4.159.190:37441
I20260807 08:47:50.887036 9649 heartbeater.cc:464] Registering TS with master...
I20260807 08:47:50.887198 9649 heartbeater.cc:511] Master 127.4.159.190:37441 requested a full tablet report, sending...
I20260807 08:47:50.887547 9182 ts_manager.cc:194] Registered new tserver with Master: d9f1d6afac8f40629b11fca74231963c (127.4.159.130:46041)
I20260807 08:47:50.887897 9182 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.159.130@KRBTEST.COM'} at 127.4.159.130:40843
WARNING: no policy specified for kudu/127.4.159.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.159.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.159.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.131.keytab.
Entry for principal kudu/127.4.159.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.131.keytab.
I20260807 08:47:50.909235 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786092453486433-4734-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786092453486433-4734-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786092453486433-4734-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786092453486433-4734-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.159.131:0
--local_ip_for_outbound_sockets=127.4.159.131
--webserver_interface=127.4.159.131
--webserver_port=0
--tserver_master_addrs=127.4.159.190:37441
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.131.keytab
--principal=kudu/127.4.159.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:42093
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:47:50.987397 9654 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:47:50.987579 9654 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:47:50.987609 9654 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:47:50.988703 9654 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:47:50.988742 9654 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:47:50.988776 9654 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.159.131
I20260807 08:47:50.990159 9654 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:42093
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786092453486433-4734-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786092453486433-4734-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.131.keytab
--principal=kudu/127.4.159.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.159.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786092453486433-4734-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.159.131
--webserver_port=0
--tserver_master_addrs=127.4.159.190:37441
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9654
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.159.131
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786092453486433-4734-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:47:50.990406 9654 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:47:50.990605 9654 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:47:50.995159 9665 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:47:50.995108 9661 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:47:50.995182 9654 server_base.cc:1034] running on GCE node
W20260807 08:47:50.995100 9663 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:47:50.995545 9654 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:47:50.995780 9654 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:47:50.996899 9654 hybrid_clock.cc:648] HybridClock initialized: now 1786092470996893 us; error 27 us; skew 500 ppm
Aug 07 08:47:50 dist-test-slave-6zbq krb5kdc[9114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092470, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:47:50.999229 9654 init.cc:377] Logged in from keytab as kudu/127.4.159.131@KRBTEST.COM (short username kudu)
I20260807 08:47:50.999612 9654 webserver.cc:460] Webserver started at http://127.4.159.131:39387/ using document root <none> and password file <none>
I20260807 08:47:50.999778 9654 fs_manager.cc:359] Metadata directory not provided
I20260807 08:47:50.999814 9654 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:47:50.999902 9654 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:47:51.000800 9654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786092453486433-4734-0/minicluster-data/ts-2/data/instance:
uuid: "04042e47e942403d85252bd71efeba7b"
format_stamp: "Formatted at 2026-08-07 08:47:50 on dist-test-slave-6zbq"
server_key: "73cb8cf805b60f58a87cede94f4e3377"
server_key_iv: "5c1d3638e19da929e160c25f4c6b472b"
server_key_version: "encryptionkey@0"
I20260807 08:47:51.001075 9654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786092453486433-4734-0/minicluster-data/ts-2/wal/instance:
uuid: "04042e47e942403d85252bd71efeba7b"
format_stamp: "Formatted at 2026-08-07 08:47:50 on dist-test-slave-6zbq"
server_key: "73cb8cf805b60f58a87cede94f4e3377"
server_key_iv: "5c1d3638e19da929e160c25f4c6b472b"
server_key_version: "encryptionkey@0"
I20260807 08:47:51.002206 9654 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 08:47:51.002887 9676 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:51.003036 9654 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:51.003132 9654 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786092453486433-4734-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786092453486433-4734-0/minicluster-data/ts-2/wal
uuid: "04042e47e942403d85252bd71efeba7b"
format_stamp: "Formatted at 2026-08-07 08:47:50 on dist-test-slave-6zbq"
server_key: "73cb8cf805b60f58a87cede94f4e3377"
server_key_iv: "5c1d3638e19da929e160c25f4c6b472b"
server_key_version: "encryptionkey@0"
I20260807 08:47:51.003202 9654 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786092453486433-4734-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786092453486433-4734-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786092453486433-4734-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:47:51.014034 9654 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:47:51.015705 9654 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:47:51.015815 9654 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:47:51.016006 9654 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:47:51.016265 9654 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:47:51.016299 9654 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:51.016316 9654 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:47:51.016327 9654 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:51.021790 9654 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.131:41339
I20260807 08:47:51.021847 9837 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.131:41339 every 8 connection(s)
I20260807 08:47:51.022094 9654 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786092453486433-4734-0/minicluster-data/ts-2/data/info.pb
I20260807 08:47:51.023468 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 9654
I20260807 08:47:51.023571 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786092453486433-4734-0/minicluster-data/ts-2/wal/instance
I20260807 08:47:51.023772 4734 external_mini_cluster.cc:1428] Setting key 59e1a6d22f9c25728256c7c36564195d
Aug 07 08:47:51 dist-test-slave-6zbq krb5kdc[9114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092470, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.131@KRBTEST.COM for kudu/127.4.159.190@KRBTEST.COM
I20260807 08:47:51.030687 9234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:51.022708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.131:41249 (local address 127.4.159.190:37441)
0807 08:47:51.022761 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:51.022763 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:51.022919 (+ 156us) server_negotiation.cc:407] Connection header received
0807 08:47:51.023418 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:51.023419 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:51.023430 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:51.023465 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:47:51.024568 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:51.025075 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:51.026008 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:51.026211 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:51.026264 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:51.028157 (+ 1893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:47:51.028159 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:47:51.028159 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:47:51.028166 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:47:51.029442 (+ 1276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:51.029790 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:51.029791 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:51.029792 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:51.029846 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:51.030050 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:51.030051 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:51.030052 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:51.030161 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:47:51.030215 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:51.030502 (+ 287us) server_negotiation.cc:299] Negotiation successful
0807 08:47:51.030585 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260807 08:47:51.030798 9840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:51.022669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:37441 (local address 127.4.159.131:41249)
0807 08:47:51.022808 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0807 08:47:51.022850 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 08:47:51.023302 (+ 452us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:47:51.023552 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:47:51.023557 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:47:51.024237 (+ 680us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:47:51.024491 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:51.024497 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:51.025163 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:51.025165 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:51.025927 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:51.025929 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:51.026277 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:51.026278 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:51.026433 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:51.026840 (+ 407us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:47:51.026847 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:47:51.028074 (+ 1227us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:47:51.029515 (+ 1441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:47:51.029522 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:47:51.029524 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:47:51.029724 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:47:51.029899 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:47:51.029899 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:47:51.029900 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:47:51.029996 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:47:51.030216 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:47:51.030222 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:47:51.030372 (+ 150us) client_negotiation.cc:769] Sending connection context
0807 08:47:51.030469 (+ 97us) client_negotiation.cc:240] Negotiation successful
0807 08:47:51.030607 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":63,"threads_started":1}
I20260807 08:47:51.031189 9838 heartbeater.cc:347] Connected to a master server at 127.4.159.190:37441
I20260807 08:47:51.031272 9838 heartbeater.cc:464] Registering TS with master...
I20260807 08:47:51.031422 9838 heartbeater.cc:511] Master 127.4.159.190:37441 requested a full tablet report, sending...
I20260807 08:47:51.031729 9182 ts_manager.cc:194] Registered new tserver with Master: 04042e47e942403d85252bd71efeba7b (127.4.159.131:41339)
I20260807 08:47:51.032076 9182 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.159.131@KRBTEST.COM'} at 127.4.159.131:41249
I20260807 08:47:51.035276 4734 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 08:47:51 dist-test-slave-6zbq krb5kdc[9114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092471, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 07 08:47:51 dist-test-slave-6zbq krb5kdc[9114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092471, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.4.159.190@KRBTEST.COM
I20260807 08:47:51.056607 9234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:51.050389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32820 (local address 127.4.159.190:37441)
0807 08:47:51.050442 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:51.050444 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:51.050516 (+ 72us) server_negotiation.cc:407] Connection header received
0807 08:47:51.050579 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:51.050580 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:51.050590 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:51.050654 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:47:51.051116 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:51.051562 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:51.052276 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:51.052458 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:51.052509 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:51.054426 (+ 1917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:47:51.054427 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:47:51.054428 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:47:51.054435 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:47:51.055618 (+ 1183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:51.055916 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:51.055917 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:51.055918 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:51.055966 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:51.056124 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:51.056125 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:51.056125 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:51.056253 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:47:51.056338 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:51.056439 (+ 101us) server_negotiation.cc:299] Negotiation successful
0807 08:47:51.056506 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
W20260807 08:47:51.056809 9182 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:32820
I20260807 08:47:51.057149 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 9266
I20260807 08:47:51.068974 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 9459
I20260807 08:47:51.082218 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 9654
I20260807 08:47:51.089048 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 9134
2026-08-07T08:47:51Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (1888 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786092453486433-4734-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:47:51 dist-test-slave-6zbq krb5kdc[9856](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:47:51 dist-test-slave-6zbq krb5kdc[9856](info): set up 2 sockets
Aug 07 08:47:51 dist-test-slave-6zbq krb5kdc[9856](info): commencing operation
krb5kdc: starting...
W20260807 08:47:52.206275 4734 mini_kdc.cc:121] Time spent starting KDC: real 1.093s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786092453486433-4734-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786092453486433-4734-0/krb5kdc/test-user.keytab.
Aug 07 08:47:52 dist-test-slave-6zbq krb5kdc[9856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092472, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:47:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:47:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.159.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.159.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.159.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab.
Entry for principal kudu/127.4.159.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab.
I20260807 08:47:52.312387 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786092453486433-4734-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786092453486433-4734-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786092453486433-4734-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.159.190:37335
--webserver_interface=127.4.159.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab
--principal=kudu/127.4.159.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:42735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.159.190:37335
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:47:52.391459 9875 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:47:52.391638 9875 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:47:52.391659 9875 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:47:52.392714 9875 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:47:52.392746 9875 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:47:52.392758 9875 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:47:52.392769 9875 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:47:52.392781 9875 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:47:52.394243 9875 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:42735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786092453486433-4734-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786092453486433-4734-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.159.190:37335
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab
--principal=kudu/127.4.159.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.159.190:37335
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.159.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9875
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786092453486433-4734-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:47:52.394454 9875 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:47:52.394620 9875 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:47:52.398936 9885 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:47:52.398988 9875 server_base.cc:1034] running on GCE node
W20260807 08:47:52.398944 9884 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:47:52.398957 9887 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:47:52.399372 9875 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:47:52.399631 9875 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:47:52.400760 9875 hybrid_clock.cc:648] HybridClock initialized: now 1786092472400748 us; error 30 us; skew 500 ppm
Aug 07 08:47:52 dist-test-slave-6zbq krb5kdc[9856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092472, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:47:52.402961 9875 init.cc:377] Logged in from keytab as kudu/127.4.159.190@KRBTEST.COM (short username kudu)
I20260807 08:47:52.403364 9875 webserver.cc:460] Webserver started at http://127.4.159.190:45341/ using document root <none> and password file <none>
I20260807 08:47:52.403554 9875 fs_manager.cc:359] Metadata directory not provided
I20260807 08:47:52.403611 9875 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:47:52.403733 9875 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:47:52.404727 9875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786092453486433-4734-0/minicluster-data/master-0/data/instance:
uuid: "26acde5a932544c0a02bc23787634020"
format_stamp: "Formatted at 2026-08-07 08:47:52 on dist-test-slave-6zbq"
server_key: "bb9a23c90b2fed169410d127b5c8930b"
server_key_iv: "82785f36c082d0edd2967a0f2f56e80a"
server_key_version: "encryptionkey@0"
I20260807 08:47:52.405037 9875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786092453486433-4734-0/minicluster-data/master-0/wal/instance:
uuid: "26acde5a932544c0a02bc23787634020"
format_stamp: "Formatted at 2026-08-07 08:47:52 on dist-test-slave-6zbq"
server_key: "bb9a23c90b2fed169410d127b5c8930b"
server_key_iv: "82785f36c082d0edd2967a0f2f56e80a"
server_key_version: "encryptionkey@0"
I20260807 08:47:52.406262 9875 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260807 08:47:52.406977 9896 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:52.407140 9875 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 08:47:52.407219 9875 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786092453486433-4734-0/minicluster-data/master-0/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786092453486433-4734-0/minicluster-data/master-0/wal
uuid: "26acde5a932544c0a02bc23787634020"
format_stamp: "Formatted at 2026-08-07 08:47:52 on dist-test-slave-6zbq"
server_key: "bb9a23c90b2fed169410d127b5c8930b"
server_key_iv: "82785f36c082d0edd2967a0f2f56e80a"
server_key_version: "encryptionkey@0"
I20260807 08:47:52.407279 9875 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786092453486433-4734-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786092453486433-4734-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786092453486433-4734-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:47:52.446045 9875 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:47:52.447804 9875 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:47:52.447940 9875 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:47:52.451815 9875 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.190:37335
I20260807 08:47:52.451848 9966 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.190:37335 every 8 connection(s)
I20260807 08:47:52.452386 9875 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
I20260807 08:47:52.453022 9967 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:47:52.455595 9967 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26acde5a932544c0a02bc23787634020: Bootstrap starting.
I20260807 08:47:52.456172 9967 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 26acde5a932544c0a02bc23787634020: Neither blocks nor log segments found. Creating new log.
I20260807 08:47:52.456413 9967 log.cc:824] T 00000000000000000000000000000000 P 26acde5a932544c0a02bc23787634020: Log is configured to *not* fsync() on all Append() calls
I20260807 08:47:52.457010 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 9875
I20260807 08:47:52.457120 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786092453486433-4734-0/minicluster-data/master-0/wal/instance
I20260807 08:47:52.457124 9967 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26acde5a932544c0a02bc23787634020: No bootstrap required, opened a new log
I20260807 08:47:52.457299 4734 external_mini_cluster.cc:1428] Setting key 91b009e32105c73cbe3afb0d9fe2b921
I20260807 08:47:52.459779 9967 raft_consensus.cc:348] T 00000000000000000000000000000000 P 26acde5a932544c0a02bc23787634020 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26acde5a932544c0a02bc23787634020" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 37335 } }
I20260807 08:47:52.459909 9967 raft_consensus.cc:374] T 00000000000000000000000000000000 P 26acde5a932544c0a02bc23787634020 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:47:52.459930 9967 raft_consensus.cc:729] T 00000000000000000000000000000000 P 26acde5a932544c0a02bc23787634020 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26acde5a932544c0a02bc23787634020, State: Initialized, Role: FOLLOWER
I20260807 08:47:52.460029 9967 consensus_queue.cc:260] T 00000000000000000000000000000000 P 26acde5a932544c0a02bc23787634020 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26acde5a932544c0a02bc23787634020" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 37335 } }
I20260807 08:47:52.460104 9967 raft_consensus.cc:388] T 00000000000000000000000000000000 P 26acde5a932544c0a02bc23787634020 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:47:52.460129 9967 raft_consensus.cc:482] T 00000000000000000000000000000000 P 26acde5a932544c0a02bc23787634020 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:47:52.460175 9967 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 26acde5a932544c0a02bc23787634020 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:47:52.460812 9967 raft_consensus.cc:504] T 00000000000000000000000000000000 P 26acde5a932544c0a02bc23787634020 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26acde5a932544c0a02bc23787634020" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 37335 } }
I20260807 08:47:52.460922 9967 leader_election.cc:302] T 00000000000000000000000000000000 P 26acde5a932544c0a02bc23787634020 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 26acde5a932544c0a02bc23787634020; no voters:
I20260807 08:47:52.461062 9967 leader_election.cc:288] T 00000000000000000000000000000000 P 26acde5a932544c0a02bc23787634020 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:47:52.461105 9975 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 26acde5a932544c0a02bc23787634020 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:47:52.461259 9975 raft_consensus.cc:686] T 00000000000000000000000000000000 P 26acde5a932544c0a02bc23787634020 [term 1 LEADER]: Becoming Leader. State: Replica: 26acde5a932544c0a02bc23787634020, State: Running, Role: LEADER
I20260807 08:47:52.461315 9967 sys_catalog.cc:564] T 00000000000000000000000000000000 P 26acde5a932544c0a02bc23787634020 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:47:52.461350 9975 consensus_queue.cc:237] T 00000000000000000000000000000000 P 26acde5a932544c0a02bc23787634020 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26acde5a932544c0a02bc23787634020" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 37335 } }
I20260807 08:47:52.461673 9975 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26acde5a932544c0a02bc23787634020 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "26acde5a932544c0a02bc23787634020" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26acde5a932544c0a02bc23787634020" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 37335 } } }
I20260807 08:47:52.461827 9975 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26acde5a932544c0a02bc23787634020 [sys.catalog]: This master's current role is: LEADER
I20260807 08:47:52.462013 9975 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26acde5a932544c0a02bc23787634020 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 26acde5a932544c0a02bc23787634020. Latest consensus state: current_term: 1 leader_uuid: "26acde5a932544c0a02bc23787634020" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26acde5a932544c0a02bc23787634020" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 37335 } } }
I20260807 08:47:52.462074 9975 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26acde5a932544c0a02bc23787634020 [sys.catalog]: This master's current role is: LEADER
I20260807 08:47:52.462189 9986 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:47:52.462721 9986 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 07 08:47:52 dist-test-slave-6zbq krb5kdc[9856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092472, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.159.190@KRBTEST.COM
I20260807 08:47:52.465013 9986 catalog_manager.cc:1347] Generated new cluster ID: ea0f4728505a4fe98a6320fc9277c533
I20260807 08:47:52.465066 9986 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:47:52.467720 9973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:52.457772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35008 (local address 127.4.159.190:37335)
0807 08:47:52.457928 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:52.457936 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:52.457964 (+ 28us) server_negotiation.cc:407] Connection header received
0807 08:47:52.458398 (+ 434us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:52.458404 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:52.458451 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:52.458556 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:47:52.459300 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:52.460234 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:52.461250 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:52.461854 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:52.461971 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:52.464407 (+ 2436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:47:52.464411 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:47:52.464412 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:47:52.464424 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:47:52.466241 (+ 1817us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:52.466664 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:52.466667 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:52.466668 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:52.466752 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:52.466931 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:52.466931 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:52.466932 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:52.467168 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:47:52.467323 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:52.467439 (+ 116us) server_negotiation.cc:299] Negotiation successful
0807 08:47:52.467562 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":61,"threads_started":1}
I20260807 08:47:52.502607 9986 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:47:52.503275 9986 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:47:52.506984 9986 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 26acde5a932544c0a02bc23787634020: Generated new TSK 0
I20260807 08:47:52.507133 9986 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.159.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.159.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.159.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.129.keytab.
Entry for principal kudu/127.4.159.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.129.keytab.
I20260807 08:47:52.534312 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786092453486433-4734-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786092453486433-4734-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786092453486433-4734-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786092453486433-4734-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.159.129:0
--local_ip_for_outbound_sockets=127.4.159.129
--webserver_interface=127.4.159.129
--webserver_port=0
--tserver_master_addrs=127.4.159.190:37335
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.129.keytab
--principal=kudu/127.4.159.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:42735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:47:52.613811 10003 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:47:52.613987 10003 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:47:52.614009 10003 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:47:52.615137 10003 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:47:52.615175 10003 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:47:52.615199 10003 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.159.129
I20260807 08:47:52.616591 10003 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:42735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786092453486433-4734-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786092453486433-4734-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.129.keytab
--principal=kudu/127.4.159.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.159.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786092453486433-4734-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.159.129
--webserver_port=0
--tserver_master_addrs=127.4.159.190:37335
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10003
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.159.129
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786092453486433-4734-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:47:52.616758 10003 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:47:52.616921 10003 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:47:52.621309 10013 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:47:52.621311 10011 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:47:52.621395 10017 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:47:52.621459 10003 server_base.cc:1034] running on GCE node
I20260807 08:47:52.621659 10003 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:47:52.621873 10003 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:47:52.623005 10003 hybrid_clock.cc:648] HybridClock initialized: now 1786092472622993 us; error 27 us; skew 500 ppm
Aug 07 08:47:52 dist-test-slave-6zbq krb5kdc[9856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092472, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:47:52.625304 10003 init.cc:377] Logged in from keytab as kudu/127.4.159.129@KRBTEST.COM (short username kudu)
I20260807 08:47:52.625694 10003 webserver.cc:460] Webserver started at http://127.4.159.129:40647/ using document root <none> and password file <none>
I20260807 08:47:52.625873 10003 fs_manager.cc:359] Metadata directory not provided
I20260807 08:47:52.625919 10003 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:47:52.626040 10003 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:47:52.626999 10003 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786092453486433-4734-0/minicluster-data/ts-0/data/instance:
uuid: "c92bd3d436ac4b51b99d722c326caa79"
format_stamp: "Formatted at 2026-08-07 08:47:52 on dist-test-slave-6zbq"
server_key: "aafba96226eb3c818a36864b72a26ab2"
server_key_iv: "7a39d4af30e80d78a462c19a9bd40919"
server_key_version: "encryptionkey@0"
I20260807 08:47:52.627321 10003 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786092453486433-4734-0/minicluster-data/ts-0/wal/instance:
uuid: "c92bd3d436ac4b51b99d722c326caa79"
format_stamp: "Formatted at 2026-08-07 08:47:52 on dist-test-slave-6zbq"
server_key: "aafba96226eb3c818a36864b72a26ab2"
server_key_iv: "7a39d4af30e80d78a462c19a9bd40919"
server_key_version: "encryptionkey@0"
I20260807 08:47:52.628479 10003 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260807 08:47:52.629086 10027 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:52.629230 10003 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 08:47:52.629307 10003 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786092453486433-4734-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786092453486433-4734-0/minicluster-data/ts-0/wal
uuid: "c92bd3d436ac4b51b99d722c326caa79"
format_stamp: "Formatted at 2026-08-07 08:47:52 on dist-test-slave-6zbq"
server_key: "aafba96226eb3c818a36864b72a26ab2"
server_key_iv: "7a39d4af30e80d78a462c19a9bd40919"
server_key_version: "encryptionkey@0"
I20260807 08:47:52.629362 10003 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786092453486433-4734-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786092453486433-4734-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786092453486433-4734-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:47:52.642244 10003 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:47:52.644167 10003 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:47:52.644292 10003 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:47:52.644479 10003 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:47:52.644737 10003 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:47:52.644768 10003 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:52.644793 10003 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:47:52.644810 10003 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:52.649976 10003 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.129:40527
I20260807 08:47:52.650036 10193 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.129:40527 every 8 connection(s)
I20260807 08:47:52.650287 10003 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786092453486433-4734-0/minicluster-data/ts-0/data/info.pb
Aug 07 08:47:52 dist-test-slave-6zbq krb5kdc[9856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092472, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.129@KRBTEST.COM for kudu/127.4.159.190@KRBTEST.COM
I20260807 08:47:52.658293 9973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:52.650918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.129:44555 (local address 127.4.159.190:37335)
0807 08:47:52.650980 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:52.650982 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:52.651131 (+ 149us) server_negotiation.cc:407] Connection header received
0807 08:47:52.651964 (+ 833us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:52.651965 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:52.651979 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:52.652030 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:47:52.652589 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:52.653089 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:52.654008 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:52.654229 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:52.654283 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:52.656106 (+ 1823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:47:52.656108 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:47:52.656108 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:47:52.656115 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:47:52.657106 (+ 991us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:52.657460 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:52.657462 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:52.657462 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:52.657515 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:52.657723 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:52.657723 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:52.657724 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:52.657831 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:47:52.657884 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:52.658120 (+ 236us) server_negotiation.cc:299] Negotiation successful
0807 08:47:52.658206 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260807 08:47:52.658329 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 10003
I20260807 08:47:52.658444 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786092453486433-4734-0/minicluster-data/ts-0/wal/instance
I20260807 08:47:52.658492 10197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:52.650874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:37335 (local address 127.4.159.129:44555)
0807 08:47:52.651029 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0807 08:47:52.651061 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0807 08:47:52.651864 (+ 803us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:47:52.652093 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:47:52.652096 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:47:52.652333 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:47:52.652501 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:52.652505 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:52.653171 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:52.653172 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:52.653925 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:52.653927 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:52.654300 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:52.654301 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:52.654438 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:52.654786 (+ 348us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:47:52.654793 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:47:52.656016 (+ 1223us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:47:52.657177 (+ 1161us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:47:52.657184 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:47:52.657186 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:47:52.657394 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:47:52.657569 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:47:52.657569 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:47:52.657570 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:47:52.657663 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:47:52.657885 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:47:52.657891 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:47:52.658009 (+ 118us) client_negotiation.cc:769] Sending connection context
0807 08:47:52.658092 (+ 83us) client_negotiation.cc:240] Negotiation successful
0807 08:47:52.658263 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":75,"threads_started":1}
I20260807 08:47:52.658632 4734 external_mini_cluster.cc:1428] Setting key 80d183480cc116aba01cac6158884098
I20260807 08:47:52.659101 10195 heartbeater.cc:347] Connected to a master server at 127.4.159.190:37335
I20260807 08:47:52.659189 10195 heartbeater.cc:464] Registering TS with master...
I20260807 08:47:52.659353 10195 heartbeater.cc:511] Master 127.4.159.190:37335 requested a full tablet report, sending...
I20260807 08:47:52.659860 9919 ts_manager.cc:194] Registered new tserver with Master: c92bd3d436ac4b51b99d722c326caa79 (127.4.159.129:40527)
I20260807 08:47:52.660446 9919 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.159.129@KRBTEST.COM'} at 127.4.159.129:44555
WARNING: no policy specified for kudu/127.4.159.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.159.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.159.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.130.keytab.
Entry for principal kudu/127.4.159.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.130.keytab.
I20260807 08:47:52.680989 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786092453486433-4734-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786092453486433-4734-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786092453486433-4734-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786092453486433-4734-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.159.130:0
--local_ip_for_outbound_sockets=127.4.159.130
--webserver_interface=127.4.159.130
--webserver_port=0
--tserver_master_addrs=127.4.159.190:37335
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.130.keytab
--principal=kudu/127.4.159.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:42735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:47:52.759544 10201 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:47:52.759720 10201 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:47:52.759742 10201 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:47:52.760799 10201 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:47:52.760833 10201 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:47:52.760856 10201 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.159.130
I20260807 08:47:52.762473 10201 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:42735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786092453486433-4734-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786092453486433-4734-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.130.keytab
--principal=kudu/127.4.159.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.159.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786092453486433-4734-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.159.130
--webserver_port=0
--tserver_master_addrs=127.4.159.190:37335
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10201
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.159.130
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786092453486433-4734-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:47:52.762641 10201 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:47:52.762804 10201 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:47:52.767216 10208 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:47:52.767202 10212 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:47:52.767207 10207 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:47:52.767457 10201 server_base.cc:1034] running on GCE node
I20260807 08:47:52.767652 10201 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:47:52.767876 10201 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:47:52.769007 10201 hybrid_clock.cc:648] HybridClock initialized: now 1786092472768987 us; error 36 us; skew 500 ppm
Aug 07 08:47:52 dist-test-slave-6zbq krb5kdc[9856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092472, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:47:52.771311 10201 init.cc:377] Logged in from keytab as kudu/127.4.159.130@KRBTEST.COM (short username kudu)
I20260807 08:47:52.771720 10201 webserver.cc:460] Webserver started at http://127.4.159.130:43619/ using document root <none> and password file <none>
I20260807 08:47:52.771903 10201 fs_manager.cc:359] Metadata directory not provided
I20260807 08:47:52.771944 10201 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:47:52.772061 10201 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:47:52.773002 10201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786092453486433-4734-0/minicluster-data/ts-1/data/instance:
uuid: "dbd6eeac792346f5a3db34506d9c1cfe"
format_stamp: "Formatted at 2026-08-07 08:47:52 on dist-test-slave-6zbq"
server_key: "dc0423ed594e4c5db3491e194cfcd476"
server_key_iv: "5ea52a819032a0336800bfd477ac5652"
server_key_version: "encryptionkey@0"
I20260807 08:47:52.773317 10201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786092453486433-4734-0/minicluster-data/ts-1/wal/instance:
uuid: "dbd6eeac792346f5a3db34506d9c1cfe"
format_stamp: "Formatted at 2026-08-07 08:47:52 on dist-test-slave-6zbq"
server_key: "dc0423ed594e4c5db3491e194cfcd476"
server_key_iv: "5ea52a819032a0336800bfd477ac5652"
server_key_version: "encryptionkey@0"
I20260807 08:47:52.774490 10201 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260807 08:47:52.775144 10220 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:52.775271 10201 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:52.775347 10201 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786092453486433-4734-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786092453486433-4734-0/minicluster-data/ts-1/wal
uuid: "dbd6eeac792346f5a3db34506d9c1cfe"
format_stamp: "Formatted at 2026-08-07 08:47:52 on dist-test-slave-6zbq"
server_key: "dc0423ed594e4c5db3491e194cfcd476"
server_key_iv: "5ea52a819032a0336800bfd477ac5652"
server_key_version: "encryptionkey@0"
I20260807 08:47:52.775413 10201 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786092453486433-4734-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786092453486433-4734-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786092453486433-4734-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:47:52.787592 10201 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:47:52.789163 10201 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:47:52.789283 10201 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:47:52.789464 10201 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:47:52.789748 10201 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:47:52.789781 10201 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:52.789808 10201 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:47:52.789824 10201 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:52.795185 10201 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.130:40539
I20260807 08:47:52.795245 10389 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.130:40539 every 8 connection(s)
I20260807 08:47:52.795490 10201 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786092453486433-4734-0/minicluster-data/ts-1/data/info.pb
Aug 07 08:47:52 dist-test-slave-6zbq krb5kdc[9856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092472, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.130@KRBTEST.COM for kudu/127.4.159.190@KRBTEST.COM
I20260807 08:47:52.803169 9973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:52.796306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.130:38287 (local address 127.4.159.190:37335)
0807 08:47:52.796348 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:52.796350 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:52.796358 (+ 8us) server_negotiation.cc:407] Connection header received
0807 08:47:52.797110 (+ 752us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:52.797111 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:52.797122 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:52.797170 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:47:52.797700 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:52.798170 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:52.799089 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:52.799285 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:52.799338 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:52.801065 (+ 1727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:47:52.801066 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:47:52.801066 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:47:52.801072 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:47:52.802172 (+ 1100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:52.802531 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:52.802531 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:52.802532 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:52.802573 (+ 41us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:52.802752 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:52.802753 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:52.802753 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:52.802840 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:47:52.802883 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:52.803055 (+ 172us) server_negotiation.cc:299] Negotiation successful
0807 08:47:52.803106 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 08:47:52.803325 10394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:52.796065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:37335 (local address 127.4.159.130:38287)
0807 08:47:52.796207 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0807 08:47:52.796239 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0807 08:47:52.797024 (+ 785us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:47:52.797232 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:47:52.797235 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:47:52.797451 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:47:52.797612 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:52.797644 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:52.798227 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:52.798227 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:52.799006 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:52.799008 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:52.799344 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:52.799345 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:52.799430 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:52.799823 (+ 393us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:47:52.799830 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:47:52.800991 (+ 1161us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:47:52.802234 (+ 1243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:47:52.802240 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:47:52.802242 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:47:52.802479 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:47:52.802634 (+ 155us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:47:52.802635 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:47:52.802635 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:47:52.802719 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:47:52.802880 (+ 161us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:47:52.802882 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:47:52.802975 (+ 93us) client_negotiation.cc:769] Sending connection context
0807 08:47:52.803041 (+ 66us) client_negotiation.cc:240] Negotiation successful
0807 08:47:52.803145 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":72,"threads_started":1}
I20260807 08:47:52.803728 10390 heartbeater.cc:347] Connected to a master server at 127.4.159.190:37335
I20260807 08:47:52.803808 10390 heartbeater.cc:464] Registering TS with master...
I20260807 08:47:52.803960 10390 heartbeater.cc:511] Master 127.4.159.190:37335 requested a full tablet report, sending...
I20260807 08:47:52.804301 9919 ts_manager.cc:194] Registered new tserver with Master: dbd6eeac792346f5a3db34506d9c1cfe (127.4.159.130:40539)
I20260807 08:47:52.804651 9919 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.159.130@KRBTEST.COM'} at 127.4.159.130:38287
I20260807 08:47:52.805390 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 10201
I20260807 08:47:52.805476 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786092453486433-4734-0/minicluster-data/ts-1/wal/instance
I20260807 08:47:52.805634 4734 external_mini_cluster.cc:1428] Setting key f62e09c7736466779963343366d6fe5c
WARNING: no policy specified for kudu/127.4.159.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.159.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.159.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.131.keytab.
Entry for principal kudu/127.4.159.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.131.keytab.
I20260807 08:47:52.826511 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786092453486433-4734-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786092453486433-4734-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786092453486433-4734-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786092453486433-4734-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.159.131:0
--local_ip_for_outbound_sockets=127.4.159.131
--webserver_interface=127.4.159.131
--webserver_port=0
--tserver_master_addrs=127.4.159.190:37335
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.131.keytab
--principal=kudu/127.4.159.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:42735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:47:52.904724 10399 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:47:52.904899 10399 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:47:52.904920 10399 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:47:52.906039 10399 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:47:52.906080 10399 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:47:52.906109 10399 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.159.131
I20260807 08:47:52.907596 10399 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:42735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786092453486433-4734-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786092453486433-4734-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.131.keytab
--principal=kudu/127.4.159.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.159.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786092453486433-4734-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.159.131
--webserver_port=0
--tserver_master_addrs=127.4.159.190:37335
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10399
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.159.131
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786092453486433-4734-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:47:52.907796 10399 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:47:52.907986 10399 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:47:52.912569 10408 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:47:52.912559 10410 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:47:52.912618 10406 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:47:52.912791 10399 server_base.cc:1034] running on GCE node
I20260807 08:47:52.912989 10399 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:47:52.913214 10399 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:47:52.914381 10399 hybrid_clock.cc:648] HybridClock initialized: now 1786092472914360 us; error 26 us; skew 500 ppm
Aug 07 08:47:52 dist-test-slave-6zbq krb5kdc[9856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092472, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:47:52.916703 10399 init.cc:377] Logged in from keytab as kudu/127.4.159.131@KRBTEST.COM (short username kudu)
I20260807 08:47:52.917093 10399 webserver.cc:460] Webserver started at http://127.4.159.131:37345/ using document root <none> and password file <none>
I20260807 08:47:52.917300 10399 fs_manager.cc:359] Metadata directory not provided
I20260807 08:47:52.917338 10399 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:47:52.917424 10399 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:47:52.918334 10399 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786092453486433-4734-0/minicluster-data/ts-2/data/instance:
uuid: "0e713ab4e63f45f9b0e5b5b078675a7e"
format_stamp: "Formatted at 2026-08-07 08:47:52 on dist-test-slave-6zbq"
server_key: "087a43142f87842b73d3ebeed7253764"
server_key_iv: "9e3d1fa968dde2c1b022e6880c51e905"
server_key_version: "encryptionkey@0"
I20260807 08:47:52.918664 10399 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786092453486433-4734-0/minicluster-data/ts-2/wal/instance:
uuid: "0e713ab4e63f45f9b0e5b5b078675a7e"
format_stamp: "Formatted at 2026-08-07 08:47:52 on dist-test-slave-6zbq"
server_key: "087a43142f87842b73d3ebeed7253764"
server_key_iv: "9e3d1fa968dde2c1b022e6880c51e905"
server_key_version: "encryptionkey@0"
I20260807 08:47:52.919773 10399 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:47:52.920375 10419 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:52.920526 10399 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:52.920604 10399 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786092453486433-4734-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786092453486433-4734-0/minicluster-data/ts-2/wal
uuid: "0e713ab4e63f45f9b0e5b5b078675a7e"
format_stamp: "Formatted at 2026-08-07 08:47:52 on dist-test-slave-6zbq"
server_key: "087a43142f87842b73d3ebeed7253764"
server_key_iv: "9e3d1fa968dde2c1b022e6880c51e905"
server_key_version: "encryptionkey@0"
I20260807 08:47:52.920656 10399 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786092453486433-4734-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786092453486433-4734-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786092453486433-4734-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:47:52.930784 10399 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:47:52.932734 10399 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:47:52.932832 10399 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:47:52.932991 10399 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:47:52.933267 10399 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:47:52.933301 10399 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:52.933320 10399 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:47:52.933343 10399 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:52.938678 10399 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.131:40945
I20260807 08:47:52.938711 10577 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.131:40945 every 8 connection(s)
I20260807 08:47:52.938973 10399 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786092453486433-4734-0/minicluster-data/ts-2/data/info.pb
I20260807 08:47:52.940402 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 10399
I20260807 08:47:52.940490 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786092453486433-4734-0/minicluster-data/ts-2/wal/instance
I20260807 08:47:52.940683 4734 external_mini_cluster.cc:1428] Setting key 2250693e05adae0159f9c1c4fd0f1d4e
Aug 07 08:47:52 dist-test-slave-6zbq krb5kdc[9856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092472, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.131@KRBTEST.COM for kudu/127.4.159.190@KRBTEST.COM
I20260807 08:47:52.947482 9973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:52.939594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.131:59239 (local address 127.4.159.190:37335)
0807 08:47:52.939639 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:52.939641 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:52.939832 (+ 191us) server_negotiation.cc:407] Connection header received
0807 08:47:52.940662 (+ 830us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:52.940664 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:52.940676 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:52.940740 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:47:52.941425 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:52.941953 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:52.942904 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:52.943092 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:52.943148 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:52.944922 (+ 1774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:47:52.944923 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:47:52.944924 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:47:52.944931 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:47:52.946263 (+ 1332us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:52.946640 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:52.946641 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:52.946641 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:52.946701 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:52.946906 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:52.946906 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:52.946907 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:52.947016 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:47:52.947069 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:52.947314 (+ 245us) server_negotiation.cc:299] Negotiation successful
0807 08:47:52.947394 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260807 08:47:52.947608 10581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:52.939566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:37335 (local address 127.4.159.131:59239)
0807 08:47:52.939728 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0807 08:47:52.939759 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0807 08:47:52.940558 (+ 799us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:47:52.940832 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:47:52.940835 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:47:52.941113 (+ 278us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:47:52.941326 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:52.941331 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:52.942021 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:52.942022 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:52.942830 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:52.942833 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:52.943157 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:52.943158 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:52.943262 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:52.943605 (+ 343us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:47:52.943612 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:47:52.944841 (+ 1229us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:47:52.946335 (+ 1494us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:47:52.946360 (+ 25us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:47:52.946362 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:47:52.946570 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:47:52.946753 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:47:52.946754 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:47:52.946754 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:47:52.946853 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:47:52.947069 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:47:52.947076 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:47:52.947203 (+ 127us) client_negotiation.cc:769] Sending connection context
0807 08:47:52.947285 (+ 82us) client_negotiation.cc:240] Negotiation successful
0807 08:47:52.947420 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":91,"threads_started":1}
I20260807 08:47:52.947973 10579 heartbeater.cc:347] Connected to a master server at 127.4.159.190:37335
I20260807 08:47:52.948050 10579 heartbeater.cc:464] Registering TS with master...
I20260807 08:47:52.948208 10579 heartbeater.cc:511] Master 127.4.159.190:37335 requested a full tablet report, sending...
I20260807 08:47:52.948544 9919 ts_manager.cc:194] Registered new tserver with Master: 0e713ab4e63f45f9b0e5b5b078675a7e (127.4.159.131:40945)
I20260807 08:47:52.948890 9919 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.159.131@KRBTEST.COM'} at 127.4.159.131:59239
I20260807 08:47:52.952135 4734 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 08:47:52 dist-test-slave-6zbq krb5kdc[9856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092472, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 08:47:52 dist-test-slave-6zbq krb5kdc[9856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092472, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.159.129@KRBTEST.COM
I20260807 08:47:52.974049 10592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:52.967271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36434 (local address 127.4.159.129:40527)
0807 08:47:52.967413 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:52.967416 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:52.967426 (+ 10us) server_negotiation.cc:407] Connection header received
0807 08:47:52.967448 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:52.967450 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:52.967461 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:52.967523 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:47:52.968058 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:52.968749 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:52.969391 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:52.969664 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:52.969722 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:52.971615 (+ 1893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:47:52.971623 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:47:52.971623 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:47:52.971629 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:47:52.973121 (+ 1492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:52.973444 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:52.973446 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:52.973447 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:52.973498 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:52.973626 (+ 128us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:52.973627 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:52.973627 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:52.973737 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:47:52.973826 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:52.973890 (+ 64us) server_negotiation.cc:299] Negotiation successful
0807 08:47:52.973947 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":62,"threads_started":1}
Aug 07 08:47:52 dist-test-slave-6zbq krb5kdc[9856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092472, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.159.190@KRBTEST.COM
I20260807 08:47:52.980943 9973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:52.975308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35012 (local address 127.4.159.190:37335)
0807 08:47:52.975345 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:52.975346 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:52.975380 (+ 34us) server_negotiation.cc:407] Connection header received
0807 08:47:52.975447 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:52.975448 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:52.975458 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:52.975500 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:47:52.975920 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:52.976364 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:52.976984 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:52.977166 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:52.977217 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:52.979109 (+ 1892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:47:52.979111 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:47:52.979111 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:47:52.979118 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:47:52.980112 (+ 994us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:52.980403 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:52.980404 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:52.980404 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:52.980447 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:52.980569 (+ 122us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:52.980570 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:52.980570 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:52.980697 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:47:52.980764 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:52.980832 (+ 68us) server_negotiation.cc:299] Negotiation successful
0807 08:47:52.980886 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
W20260807 08:47:52.981144 9919 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35012
I20260807 08:47:52.981431 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 10003
I20260807 08:47:52.992441 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 10201
I20260807 08:47:53.004549 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 10399
I20260807 08:47:53.018849 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 9875
2026-08-07T08:47:53Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (1925 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786092453486433-4734-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:47:53 dist-test-slave-6zbq krb5kdc[10600](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:47:53 dist-test-slave-6zbq krb5kdc[10600](info): set up 2 sockets
Aug 07 08:47:53 dist-test-slave-6zbq krb5kdc[10600](info): commencing operation
krb5kdc: starting...
W20260807 08:47:54.135871 4734 mini_kdc.cc:121] Time spent starting KDC: real 1.097s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786092453486433-4734-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786092453486433-4734-0/krb5kdc/test-user.keytab.
Aug 07 08:47:54 dist-test-slave-6zbq krb5kdc[10600](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092474, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:47:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:47:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.159.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.159.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.159.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab.
Entry for principal kudu/127.4.159.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab.
I20260807 08:47:54.244151 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786092453486433-4734-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786092453486433-4734-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786092453486433-4734-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.159.190:37935
--webserver_interface=127.4.159.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab
--principal=kudu/127.4.159.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:46513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.159.190:37935
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:47:54.322687 10617 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:47:54.322885 10617 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:47:54.322914 10617 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:47:54.324029 10617 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:47:54.324066 10617 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:47:54.324084 10617 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:47:54.324100 10617 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:47:54.324115 10617 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:47:54.325567 10617 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:46513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786092453486433-4734-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786092453486433-4734-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.159.190:37935
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab
--principal=kudu/127.4.159.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.159.190:37935
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.159.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10617
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786092453486433-4734-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:47:54.325802 10617 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:47:54.325991 10617 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:47:54.330387 10627 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:47:54.330374 10625 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:47:54.330365 10629 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:47:54.330430 10617 server_base.cc:1034] running on GCE node
I20260807 08:47:54.330833 10617 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:47:54.331082 10617 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:47:54.332218 10617 hybrid_clock.cc:648] HybridClock initialized: now 1786092474332197 us; error 33 us; skew 500 ppm
Aug 07 08:47:54 dist-test-slave-6zbq krb5kdc[10600](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092474, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:47:54.337834 10617 init.cc:377] Logged in from keytab as kudu/127.4.159.190@KRBTEST.COM (short username kudu)
I20260807 08:47:54.338140 10617 fs_manager.cc:359] Metadata directory not provided
I20260807 08:47:54.338191 10617 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:47:54.338312 10617 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:47:54.339300 10617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786092453486433-4734-0/minicluster-data/master-0/data/instance:
uuid: "6194428e451d438ebd95dc7e042dacc9"
format_stamp: "Formatted at 2026-08-07 08:47:54 on dist-test-slave-6zbq"
server_key: "7cb978e47fde4b71b41ccd90ed43ffde"
server_key_iv: "21a6212cd84b4da043efd177261c7f54"
server_key_version: "encryptionkey@0"
I20260807 08:47:54.339623 10617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786092453486433-4734-0/minicluster-data/master-0/wal/instance:
uuid: "6194428e451d438ebd95dc7e042dacc9"
format_stamp: "Formatted at 2026-08-07 08:47:54 on dist-test-slave-6zbq"
server_key: "7cb978e47fde4b71b41ccd90ed43ffde"
server_key_iv: "21a6212cd84b4da043efd177261c7f54"
server_key_version: "encryptionkey@0"
I20260807 08:47:54.340809 10617 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 08:47:54.341449 10635 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:54.341599 10617 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 08:47:54.341683 10617 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786092453486433-4734-0/minicluster-data/master-0/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786092453486433-4734-0/minicluster-data/master-0/wal
uuid: "6194428e451d438ebd95dc7e042dacc9"
format_stamp: "Formatted at 2026-08-07 08:47:54 on dist-test-slave-6zbq"
server_key: "7cb978e47fde4b71b41ccd90ed43ffde"
server_key_iv: "21a6212cd84b4da043efd177261c7f54"
server_key_version: "encryptionkey@0"
I20260807 08:47:54.341748 10617 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786092453486433-4734-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786092453486433-4734-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786092453486433-4734-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:47:54.359750 10617 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:47:54.361758 10617 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:47:54.361889 10617 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:47:54.365567 10617 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.190:37935
I20260807 08:47:54.365609 10711 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.190:37935 every 8 connection(s)
I20260807 08:47:54.365842 10617 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
I20260807 08:47:54.366894 10712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:47:54.368057 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 10617
I20260807 08:47:54.368155 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786092453486433-4734-0/minicluster-data/master-0/wal/instance
I20260807 08:47:54.368315 4734 external_mini_cluster.cc:1428] Setting key 569352ce55f4615b9e36e7bac769d5f4
I20260807 08:47:54.369383 10712 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6194428e451d438ebd95dc7e042dacc9: Bootstrap starting.
I20260807 08:47:54.370036 10712 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6194428e451d438ebd95dc7e042dacc9: Neither blocks nor log segments found. Creating new log.
I20260807 08:47:54.370359 10712 log.cc:824] T 00000000000000000000000000000000 P 6194428e451d438ebd95dc7e042dacc9: Log is configured to *not* fsync() on all Append() calls
I20260807 08:47:54.371062 10712 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6194428e451d438ebd95dc7e042dacc9: No bootstrap required, opened a new log
I20260807 08:47:54.373095 10712 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6194428e451d438ebd95dc7e042dacc9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6194428e451d438ebd95dc7e042dacc9" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 37935 } }
I20260807 08:47:54.373231 10712 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6194428e451d438ebd95dc7e042dacc9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:47:54.373258 10712 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6194428e451d438ebd95dc7e042dacc9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6194428e451d438ebd95dc7e042dacc9, State: Initialized, Role: FOLLOWER
I20260807 08:47:54.373410 10712 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6194428e451d438ebd95dc7e042dacc9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6194428e451d438ebd95dc7e042dacc9" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 37935 } }
I20260807 08:47:54.373499 10712 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6194428e451d438ebd95dc7e042dacc9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:47:54.373538 10712 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6194428e451d438ebd95dc7e042dacc9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:47:54.373584 10712 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6194428e451d438ebd95dc7e042dacc9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:47:54.374243 10712 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6194428e451d438ebd95dc7e042dacc9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6194428e451d438ebd95dc7e042dacc9" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 37935 } }
I20260807 08:47:54.374374 10712 leader_election.cc:302] T 00000000000000000000000000000000 P 6194428e451d438ebd95dc7e042dacc9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6194428e451d438ebd95dc7e042dacc9; no voters:
I20260807 08:47:54.374522 10712 leader_election.cc:288] T 00000000000000000000000000000000 P 6194428e451d438ebd95dc7e042dacc9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:47:54.374563 10720 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6194428e451d438ebd95dc7e042dacc9 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:47:54.374742 10712 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6194428e451d438ebd95dc7e042dacc9 [sys.catalog]: configured and running, proceeding with master startup.
Aug 07 08:47:54 dist-test-slave-6zbq krb5kdc[10600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092474, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.159.190@KRBTEST.COM
I20260807 08:47:54.374735 10720 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6194428e451d438ebd95dc7e042dacc9 [term 1 LEADER]: Becoming Leader. State: Replica: 6194428e451d438ebd95dc7e042dacc9, State: Running, Role: LEADER
I20260807 08:47:54.375480 10720 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6194428e451d438ebd95dc7e042dacc9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6194428e451d438ebd95dc7e042dacc9" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 37935 } }
I20260807 08:47:54.376035 10722 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6194428e451d438ebd95dc7e042dacc9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6194428e451d438ebd95dc7e042dacc9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6194428e451d438ebd95dc7e042dacc9" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 37935 } } }
I20260807 08:47:54.376178 10722 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6194428e451d438ebd95dc7e042dacc9 [sys.catalog]: This master's current role is: LEADER
I20260807 08:47:54.376300 10721 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6194428e451d438ebd95dc7e042dacc9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6194428e451d438ebd95dc7e042dacc9. Latest consensus state: current_term: 1 leader_uuid: "6194428e451d438ebd95dc7e042dacc9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6194428e451d438ebd95dc7e042dacc9" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 37935 } } }
I20260807 08:47:54.376485 10739 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:47:54.377159 10721 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6194428e451d438ebd95dc7e042dacc9 [sys.catalog]: This master's current role is: LEADER
I20260807 08:47:54.377857 10739 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:47:54.378640 10717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:54.368765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50728 (local address 127.4.159.190:37935)
0807 08:47:54.368959 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:54.368970 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:54.369004 (+ 34us) server_negotiation.cc:407] Connection header received
0807 08:47:54.369507 (+ 503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:54.369516 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:54.369572 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:54.369744 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:47:54.370275 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:54.371535 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:54.372239 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:54.372895 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:54.373002 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:54.375492 (+ 2490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:47:54.375496 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:47:54.375497 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:47:54.375509 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:47:54.377080 (+ 1571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:54.377416 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:54.377420 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:54.377420 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:54.377527 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:54.377741 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:54.377742 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:54.377742 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:54.378028 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:47:54.378169 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:54.378290 (+ 121us) server_negotiation.cc:299] Negotiation successful
0807 08:47:54.378450 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":99,"threads_started":1}
I20260807 08:47:54.379247 10739 catalog_manager.cc:1347] Generated new cluster ID: 254f1a2f0f1f4d79ada49ea0eb8ed543
I20260807 08:47:54.379297 10739 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:47:54.394642 10739 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:47:54.395153 10739 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:47:54.399904 10739 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6194428e451d438ebd95dc7e042dacc9: Generated new TSK 0
I20260807 08:47:54.400044 10739 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.159.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.159.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.159.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.129.keytab.
Entry for principal kudu/127.4.159.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.129.keytab.
I20260807 08:47:54.422816 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786092453486433-4734-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786092453486433-4734-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786092453486433-4734-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786092453486433-4734-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.159.129:0
--local_ip_for_outbound_sockets=127.4.159.129
--webserver_interface=127.4.159.129
--webserver_port=0
--tserver_master_addrs=127.4.159.190:37935
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.129.keytab
--principal=kudu/127.4.159.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:46513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:47:54.501973 10748 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:47:54.502152 10748 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:47:54.502174 10748 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:47:54.503448 10748 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:47:54.503507 10748 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:47:54.503539 10748 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.159.129
I20260807 08:47:54.505038 10748 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:46513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786092453486433-4734-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786092453486433-4734-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.129.keytab
--principal=kudu/127.4.159.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.159.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786092453486433-4734-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.159.129
--webserver_port=0
--tserver_master_addrs=127.4.159.190:37935
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10748
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.159.129
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786092453486433-4734-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:47:54.505250 10748 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:47:54.505450 10748 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:47:54.510087 10759 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:47:54.510087 10757 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:47:54.510200 10748 server_base.cc:1034] running on GCE node
W20260807 08:47:54.510154 10756 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:47:54.510486 10748 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:47:54.510707 10748 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:47:54.511864 10748 hybrid_clock.cc:648] HybridClock initialized: now 1786092474511850 us; error 28 us; skew 500 ppm
Aug 07 08:47:54 dist-test-slave-6zbq krb5kdc[10600](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092474, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:47:54.514118 10748 init.cc:377] Logged in from keytab as kudu/127.4.159.129@KRBTEST.COM (short username kudu)
I20260807 08:47:54.514456 10748 fs_manager.cc:359] Metadata directory not provided
I20260807 08:47:54.514508 10748 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:47:54.514638 10748 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:47:54.515565 10748 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786092453486433-4734-0/minicluster-data/ts-0/data/instance:
uuid: "aa5a9b62776a40f7ac603cb55c978167"
format_stamp: "Formatted at 2026-08-07 08:47:54 on dist-test-slave-6zbq"
server_key: "a63c4b46888e63699b1f8d43de6c68bd"
server_key_iv: "9c82a23dc6bb368e6c8f4acf072781d4"
server_key_version: "encryptionkey@0"
I20260807 08:47:54.515895 10748 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786092453486433-4734-0/minicluster-data/ts-0/wal/instance:
uuid: "aa5a9b62776a40f7ac603cb55c978167"
format_stamp: "Formatted at 2026-08-07 08:47:54 on dist-test-slave-6zbq"
server_key: "a63c4b46888e63699b1f8d43de6c68bd"
server_key_iv: "9c82a23dc6bb368e6c8f4acf072781d4"
server_key_version: "encryptionkey@0"
I20260807 08:47:54.517141 10748 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 08:47:54.517767 10768 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:54.517935 10748 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 08:47:54.518015 10748 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786092453486433-4734-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786092453486433-4734-0/minicluster-data/ts-0/wal
uuid: "aa5a9b62776a40f7ac603cb55c978167"
format_stamp: "Formatted at 2026-08-07 08:47:54 on dist-test-slave-6zbq"
server_key: "a63c4b46888e63699b1f8d43de6c68bd"
server_key_iv: "9c82a23dc6bb368e6c8f4acf072781d4"
server_key_version: "encryptionkey@0"
I20260807 08:47:54.518074 10748 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786092453486433-4734-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786092453486433-4734-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786092453486433-4734-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:47:54.535938 10748 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:47:54.537559 10748 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:47:54.537680 10748 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:47:54.537863 10748 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:47:54.538128 10748 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:47:54.538168 10748 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:54.538194 10748 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:47:54.538210 10748 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:54.543529 10748 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.129:44231
I20260807 08:47:54.543567 10936 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.129:44231 every 8 connection(s)
I20260807 08:47:54.543781 10748 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786092453486433-4734-0/minicluster-data/ts-0/data/info.pb
I20260807 08:47:54.546855 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 10748
I20260807 08:47:54.546940 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786092453486433-4734-0/minicluster-data/ts-0/wal/instance
I20260807 08:47:54.547101 4734 external_mini_cluster.cc:1428] Setting key 8c16616ca2a44943b135a769f4464297
Aug 07 08:47:54 dist-test-slave-6zbq krb5kdc[10600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092474, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.129@KRBTEST.COM for kudu/127.4.159.190@KRBTEST.COM
I20260807 08:47:54.552249 10717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:54.544362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.129:36039 (local address 127.4.159.190:37935)
0807 08:47:54.544405 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:54.544408 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:54.544643 (+ 235us) server_negotiation.cc:407] Connection header received
0807 08:47:54.545067 (+ 424us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:54.545068 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:54.545078 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:54.545118 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:47:54.546130 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:54.546641 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:54.547727 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:54.547991 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:54.548061 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:54.549909 (+ 1848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:47:54.549910 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:47:54.549910 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:47:54.549918 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:47:54.551057 (+ 1139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:54.551409 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:54.551410 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:54.551411 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:54.551462 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:54.551677 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:54.551678 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:54.551679 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:54.551780 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:47:54.551833 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:54.552076 (+ 243us) server_negotiation.cc:299] Negotiation successful
0807 08:47:54.552161 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260807 08:47:54.552397 10940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:54.544351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:37935 (local address 127.4.159.129:36039)
0807 08:47:54.544517 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0807 08:47:54.544549 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0807 08:47:54.544979 (+ 430us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:47:54.545172 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:47:54.545174 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:47:54.545463 (+ 289us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:47:54.546055 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:54.546061 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:54.546722 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:54.546723 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:54.547671 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:54.547674 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:54.548059 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:54.548060 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:54.548150 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:54.548550 (+ 400us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:47:54.548559 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:47:54.549826 (+ 1267us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:47:54.551131 (+ 1305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:47:54.551138 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:47:54.551140 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:47:54.551345 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:47:54.551516 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:47:54.551517 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:47:54.551517 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:47:54.551622 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:47:54.551836 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:47:54.551843 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:47:54.551966 (+ 123us) client_negotiation.cc:769] Sending connection context
0807 08:47:54.552049 (+ 83us) client_negotiation.cc:240] Negotiation successful
0807 08:47:54.552201 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":98,"threads_started":1}
I20260807 08:47:54.552827 10938 heartbeater.cc:347] Connected to a master server at 127.4.159.190:37935
I20260807 08:47:54.552906 10938 heartbeater.cc:464] Registering TS with master...
I20260807 08:47:54.553037 10938 heartbeater.cc:511] Master 127.4.159.190:37935 requested a full tablet report, sending...
I20260807 08:47:54.553515 10665 ts_manager.cc:194] Registered new tserver with Master: aa5a9b62776a40f7ac603cb55c978167 (127.4.159.129:44231)
I20260807 08:47:54.554105 10665 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.159.129@KRBTEST.COM'} at 127.4.159.129:36039
WARNING: no policy specified for kudu/127.4.159.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.159.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.159.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.130.keytab.
Entry for principal kudu/127.4.159.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.130.keytab.
I20260807 08:47:54.569767 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786092453486433-4734-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786092453486433-4734-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786092453486433-4734-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786092453486433-4734-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.159.130:0
--local_ip_for_outbound_sockets=127.4.159.130
--webserver_interface=127.4.159.130
--webserver_port=0
--tserver_master_addrs=127.4.159.190:37935
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.130.keytab
--principal=kudu/127.4.159.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:46513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:47:54.649009 10944 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:47:54.649188 10944 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:47:54.649209 10944 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:47:54.650264 10944 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:47:54.650295 10944 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:47:54.650319 10944 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.159.130
I20260807 08:47:54.651780 10944 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:46513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786092453486433-4734-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786092453486433-4734-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.130.keytab
--principal=kudu/127.4.159.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.159.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786092453486433-4734-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.159.130
--webserver_port=0
--tserver_master_addrs=127.4.159.190:37935
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10944
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.159.130
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786092453486433-4734-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:47:54.651947 10944 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:47:54.652110 10944 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:47:54.656548 10951 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:47:54.656553 10956 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:47:54.656746 10944 server_base.cc:1034] running on GCE node
W20260807 08:47:54.656553 10952 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:47:54.656988 10944 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:47:54.657192 10944 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:47:54.658332 10944 hybrid_clock.cc:648] HybridClock initialized: now 1786092474658320 us; error 26 us; skew 500 ppm
Aug 07 08:47:54 dist-test-slave-6zbq krb5kdc[10600](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092474, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:47:54.660566 10944 init.cc:377] Logged in from keytab as kudu/127.4.159.130@KRBTEST.COM (short username kudu)
I20260807 08:47:54.660852 10944 fs_manager.cc:359] Metadata directory not provided
I20260807 08:47:54.660900 10944 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:47:54.661013 10944 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:47:54.661948 10944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786092453486433-4734-0/minicluster-data/ts-1/data/instance:
uuid: "cd382bda5f714af5802c6308f570fb4e"
format_stamp: "Formatted at 2026-08-07 08:47:54 on dist-test-slave-6zbq"
server_key: "46a3edf12e391d22a273f01496f2cef3"
server_key_iv: "3713308a9133338f1a9bb4a38a9a8dd8"
server_key_version: "encryptionkey@0"
I20260807 08:47:54.662258 10944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786092453486433-4734-0/minicluster-data/ts-1/wal/instance:
uuid: "cd382bda5f714af5802c6308f570fb4e"
format_stamp: "Formatted at 2026-08-07 08:47:54 on dist-test-slave-6zbq"
server_key: "46a3edf12e391d22a273f01496f2cef3"
server_key_iv: "3713308a9133338f1a9bb4a38a9a8dd8"
server_key_version: "encryptionkey@0"
I20260807 08:47:54.663448 10944 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 08:47:54.664079 10965 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:54.664251 10944 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 08:47:54.664330 10944 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786092453486433-4734-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786092453486433-4734-0/minicluster-data/ts-1/wal
uuid: "cd382bda5f714af5802c6308f570fb4e"
format_stamp: "Formatted at 2026-08-07 08:47:54 on dist-test-slave-6zbq"
server_key: "46a3edf12e391d22a273f01496f2cef3"
server_key_iv: "3713308a9133338f1a9bb4a38a9a8dd8"
server_key_version: "encryptionkey@0"
I20260807 08:47:54.664387 10944 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786092453486433-4734-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786092453486433-4734-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786092453486433-4734-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:47:54.677827 10944 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:47:54.680205 10944 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:47:54.680320 10944 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:47:54.680512 10944 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:47:54.680771 10944 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:47:54.680802 10944 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:54.680830 10944 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:47:54.680852 10944 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:54.686005 10944 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.130:38293
I20260807 08:47:54.686060 11124 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.130:38293 every 8 connection(s)
I20260807 08:47:54.686270 10944 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786092453486433-4734-0/minicluster-data/ts-1/data/info.pb
Aug 07 08:47:54 dist-test-slave-6zbq krb5kdc[10600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092474, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.130@KRBTEST.COM for kudu/127.4.159.190@KRBTEST.COM
I20260807 08:47:54.693918 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 10944
I20260807 08:47:54.694010 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786092453486433-4734-0/minicluster-data/ts-1/wal/instance
I20260807 08:47:54.694100 10717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:54.686881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.130:38677 (local address 127.4.159.190:37935)
0807 08:47:54.686919 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:54.686921 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:54.687105 (+ 184us) server_negotiation.cc:407] Connection header received
0807 08:47:54.687556 (+ 451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:54.687557 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:54.687569 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:54.687631 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:47:54.688463 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:54.688958 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:54.689877 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:54.690074 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:54.690129 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:54.691917 (+ 1788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:47:54.691919 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:47:54.691919 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:47:54.691927 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:47:54.692931 (+ 1004us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:54.693277 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:54.693278 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:54.693279 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:54.693329 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:54.693529 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:54.693530 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:54.693530 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:54.693642 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:47:54.693698 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:54.693934 (+ 236us) server_negotiation.cc:299] Negotiation successful
0807 08:47:54.694013 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260807 08:47:54.694190 4734 external_mini_cluster.cc:1428] Setting key 6c89c7db041337088859da3ebcd8e4d9
I20260807 08:47:54.694252 11128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:54.686853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:37935 (local address 127.4.159.130:38677)
0807 08:47:54.687007 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0807 08:47:54.687036 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0807 08:47:54.687466 (+ 430us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:47:54.687698 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:47:54.687701 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:47:54.687930 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:47:54.688376 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:54.688380 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:54.689036 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:54.689037 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:54.689797 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:54.689800 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:54.690145 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:54.690146 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:54.690259 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:54.690628 (+ 369us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:47:54.690635 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:47:54.691834 (+ 1199us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:47:54.693002 (+ 1168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:47:54.693008 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:47:54.693010 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:47:54.693213 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:47:54.693381 (+ 168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:47:54.693381 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:47:54.693382 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:47:54.693475 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:47:54.693696 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:47:54.693703 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:47:54.693824 (+ 121us) client_negotiation.cc:769] Sending connection context
0807 08:47:54.693906 (+ 82us) client_negotiation.cc:240] Negotiation successful
0807 08:47:54.694045 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":76,"threads_started":1}
I20260807 08:47:54.694707 11125 heartbeater.cc:347] Connected to a master server at 127.4.159.190:37935
I20260807 08:47:54.694788 11125 heartbeater.cc:464] Registering TS with master...
I20260807 08:47:54.694931 11125 heartbeater.cc:511] Master 127.4.159.190:37935 requested a full tablet report, sending...
I20260807 08:47:54.695278 10664 ts_manager.cc:194] Registered new tserver with Master: cd382bda5f714af5802c6308f570fb4e (127.4.159.130:38293)
I20260807 08:47:54.695708 10664 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.159.130@KRBTEST.COM'} at 127.4.159.130:38677
WARNING: no policy specified for kudu/127.4.159.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.159.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.159.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.131.keytab.
Entry for principal kudu/127.4.159.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.131.keytab.
I20260807 08:47:54.716943 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786092453486433-4734-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786092453486433-4734-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786092453486433-4734-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786092453486433-4734-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.159.131:0
--local_ip_for_outbound_sockets=127.4.159.131
--webserver_interface=127.4.159.131
--webserver_port=0
--tserver_master_addrs=127.4.159.190:37935
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.131.keytab
--principal=kudu/127.4.159.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:46513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:47:54.798823 11137 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:47:54.799041 11137 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:47:54.799067 11137 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:47:54.800233 11137 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:47:54.800276 11137 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:47:54.800305 11137 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.159.131
I20260807 08:47:54.801815 11137 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:46513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786092453486433-4734-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786092453486433-4734-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.131.keytab
--principal=kudu/127.4.159.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.159.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786092453486433-4734-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.159.131
--webserver_port=0
--tserver_master_addrs=127.4.159.190:37935
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11137
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.159.131
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786092453486433-4734-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:47:54.802031 11137 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:47:54.802235 11137 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:47:54.806712 11147 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:47:54.806721 11142 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:47:54.806806 11137 server_base.cc:1034] running on GCE node
W20260807 08:47:54.806754 11143 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:47:54.807101 11137 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:47:54.807292 11137 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:47:54.808429 11137 hybrid_clock.cc:648] HybridClock initialized: now 1786092474808414 us; error 26 us; skew 500 ppm
Aug 07 08:47:54 dist-test-slave-6zbq krb5kdc[10600](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092474, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:47:54.810685 11137 init.cc:377] Logged in from keytab as kudu/127.4.159.131@KRBTEST.COM (short username kudu)
I20260807 08:47:54.810988 11137 fs_manager.cc:359] Metadata directory not provided
I20260807 08:47:54.811038 11137 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:47:54.811158 11137 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:47:54.812081 11137 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786092453486433-4734-0/minicluster-data/ts-2/data/instance:
uuid: "0849bc73ccd44fca824ee36227576769"
format_stamp: "Formatted at 2026-08-07 08:47:54 on dist-test-slave-6zbq"
server_key: "1df2e257b130618b8458720cf6de7ee9"
server_key_iv: "ea4a5963cdb49331b7d7404e6b43b770"
server_key_version: "encryptionkey@0"
I20260807 08:47:54.812394 11137 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786092453486433-4734-0/minicluster-data/ts-2/wal/instance:
uuid: "0849bc73ccd44fca824ee36227576769"
format_stamp: "Formatted at 2026-08-07 08:47:54 on dist-test-slave-6zbq"
server_key: "1df2e257b130618b8458720cf6de7ee9"
server_key_iv: "ea4a5963cdb49331b7d7404e6b43b770"
server_key_version: "encryptionkey@0"
I20260807 08:47:54.813520 11137 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 08:47:54.814119 11154 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:54.814265 11137 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 08:47:54.814361 11137 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786092453486433-4734-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786092453486433-4734-0/minicluster-data/ts-2/wal
uuid: "0849bc73ccd44fca824ee36227576769"
format_stamp: "Formatted at 2026-08-07 08:47:54 on dist-test-slave-6zbq"
server_key: "1df2e257b130618b8458720cf6de7ee9"
server_key_iv: "ea4a5963cdb49331b7d7404e6b43b770"
server_key_version: "encryptionkey@0"
I20260807 08:47:54.814426 11137 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786092453486433-4734-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786092453486433-4734-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786092453486433-4734-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:47:54.825202 11137 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:47:54.826761 11137 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:47:54.826885 11137 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:47:54.827064 11137 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:47:54.827324 11137 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:47:54.827355 11137 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:54.827380 11137 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:47:54.827397 11137 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:54.832607 11137 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.131:41539
I20260807 08:47:54.832635 11331 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.131:41539 every 8 connection(s)
I20260807 08:47:54.832849 11137 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786092453486433-4734-0/minicluster-data/ts-2/data/info.pb
Aug 07 08:47:54 dist-test-slave-6zbq krb5kdc[10600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092474, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.131@KRBTEST.COM for kudu/127.4.159.190@KRBTEST.COM
I20260807 08:47:54.840451 10717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:54.833438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.131:34557 (local address 127.4.159.190:37935)
0807 08:47:54.833481 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:54.833483 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:54.833651 (+ 168us) server_negotiation.cc:407] Connection header received
0807 08:47:54.834029 (+ 378us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:54.834030 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:54.834039 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:54.834086 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:47:54.834925 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:54.835416 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:54.836329 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:54.836523 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:54.836575 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:54.838281 (+ 1706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:47:54.838282 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:47:54.838282 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:47:54.838289 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:47:54.839483 (+ 1194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:54.839784 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:54.839784 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:54.839785 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:54.839829 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:54.839997 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:54.839997 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:54.839997 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:54.840085 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:47:54.840144 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:54.840334 (+ 190us) server_negotiation.cc:299] Negotiation successful
0807 08:47:54.840391 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260807 08:47:54.840587 11335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:54.833391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:37935 (local address 127.4.159.131:34557)
0807 08:47:54.833535 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0807 08:47:54.833571 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 08:47:54.833959 (+ 388us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:47:54.834141 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:47:54.834144 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:47:54.834404 (+ 260us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:47:54.834858 (+ 454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:54.834863 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:54.835490 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:54.835491 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:54.836248 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:54.836251 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:54.836580 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:54.836580 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:54.836674 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:54.837028 (+ 354us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:47:54.837035 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:47:54.838201 (+ 1166us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:47:54.839539 (+ 1338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:47:54.839542 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:47:54.839544 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:47:54.839733 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:47:54.839871 (+ 138us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:47:54.839872 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:47:54.839872 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:47:54.839958 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:47:54.840144 (+ 186us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:47:54.840147 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:47:54.840243 (+ 96us) client_negotiation.cc:769] Sending connection context
0807 08:47:54.840316 (+ 73us) client_negotiation.cc:240] Negotiation successful
0807 08:47:54.840417 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":65,"threads_started":1}
I20260807 08:47:54.841029 11332 heartbeater.cc:347] Connected to a master server at 127.4.159.190:37935
I20260807 08:47:54.841120 11332 heartbeater.cc:464] Registering TS with master...
I20260807 08:47:54.841248 11332 heartbeater.cc:511] Master 127.4.159.190:37935 requested a full tablet report, sending...
I20260807 08:47:54.841254 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 11137
I20260807 08:47:54.841333 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786092453486433-4734-0/minicluster-data/ts-2/wal/instance
I20260807 08:47:54.841518 4734 external_mini_cluster.cc:1428] Setting key 37d8c87d9b1a4ba1ae725826dcf454c3
I20260807 08:47:54.841681 10664 ts_manager.cc:194] Registered new tserver with Master: 0849bc73ccd44fca824ee36227576769 (127.4.159.131:41539)
I20260807 08:47:54.842116 10664 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.159.131@KRBTEST.COM'} at 127.4.159.131:34557
I20260807 08:47:54.842329 4734 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 08:47:54.848608 10717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:54.843504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50734 (local address 127.4.159.190:37935)
0807 08:47:54.843540 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:54.843542 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:54.843649 (+ 107us) server_negotiation.cc:407] Connection header received
0807 08:47:54.843709 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:54.843710 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:54.843719 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:54.843769 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:47:54.844304 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:54.844768 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:54.845456 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:54.845649 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:54.845700 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:54.846687 (+ 987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:47:54.846689 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:47:54.846689 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:47:54.846695 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:47:54.847626 (+ 931us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:54.847913 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:54.847914 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:54.847914 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:54.847964 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:54.848135 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:54.848136 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:54.848136 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:54.848261 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:47:54.848343 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:54.848462 (+ 119us) server_negotiation.cc:299] Negotiation successful
0807 08:47:54.848529 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260807 08:47:54.849714 10664 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50734:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 08:47:54.851198 10664 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:47:54.856777 11345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:54.852939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.129:44231 (local address 127.0.0.1:55632)
0807 08:47:54.853480 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0807 08:47:54.853487 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0807 08:47:54.853521 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:47:54.853684 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:47:54.853686 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:47:54.853687 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:47:54.853732 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:54.853734 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:54.855070 (+ 1336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:54.855071 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:54.855823 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:54.855826 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:54.856509 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:54.856510 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:54.856596 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:54.856606 (+ 10us) client_negotiation.cc:769] Sending connection context
0807 08:47:54.856662 (+ 56us) client_negotiation.cc:240] Negotiation successful
0807 08:47:54.856710 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":531,"thread_start_us":46,"threads_started":1}
I20260807 08:47:54.856801 11349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:54.853064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55632 (local address 127.4.159.129:44231)
0807 08:47:54.853264 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:54.853269 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:54.853538 (+ 269us) server_negotiation.cc:407] Connection header received
0807 08:47:54.853581 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:54.853583 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:54.853597 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:54.853684 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:47:54.853822 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:54.854956 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:54.855919 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:54.856428 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:54.856491 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:54.856544 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:54.856685 (+ 141us) server_negotiation.cc:299] Negotiation successful
0807 08:47:54.856727 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":80,"threads_started":1}
I20260807 08:47:54.857013 11350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:54.852925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.131:41539 (local address 127.0.0.1:48416)
0807 08:47:54.853273 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0807 08:47:54.853309 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 08:47:54.853380 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:47:54.853776 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:47:54.853777 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:47:54.853780 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:47:54.853843 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:54.853845 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:54.854778 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:54.854779 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:54.855974 (+ 1195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:54.855976 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:54.856812 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:54.856814 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:54.856875 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:54.856879 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 08:47:54.856909 (+ 30us) client_negotiation.cc:240] Negotiation successful
0807 08:47:54.856946 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":69,"threads_started":1}
I20260807 08:47:54.857205 11346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:54.853125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.130:38293 (local address 127.0.0.1:42188)
0807 08:47:54.853568 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0807 08:47:54.853574 (+ 6us) client_negotiation.cc:174] Beginning negotiation
0807 08:47:54.853616 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:47:54.853768 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:47:54.853769 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:47:54.853771 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:47:54.853810 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:54.853812 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:54.855108 (+ 1296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:54.855109 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:54.856137 (+ 1028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:54.856139 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:54.856976 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:54.856977 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:54.857040 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:54.857044 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 08:47:54.857107 (+ 63us) client_negotiation.cc:240] Negotiation successful
0807 08:47:54.857144 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":73,"threads_started":1}
I20260807 08:47:54.857421 11351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:54.853209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42188 (local address 127.4.159.130:38293)
0807 08:47:54.853353 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:54.853357 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:54.853624 (+ 267us) server_negotiation.cc:407] Connection header received
0807 08:47:54.853668 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:54.853670 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:54.853684 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:54.853753 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:47:54.853891 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:54.855011 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:54.856216 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:54.856904 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:54.856963 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:54.857014 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:54.857296 (+ 282us) server_negotiation.cc:299] Negotiation successful
0807 08:47:54.857341 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":87,"threads_started":1}
I20260807 08:47:54.857704 11348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:54.852978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48416 (local address 127.4.159.131:41539)
0807 08:47:54.853144 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:54.853148 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:54.853366 (+ 218us) server_negotiation.cc:407] Connection header received
0807 08:47:54.853458 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:54.853460 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:54.853470 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:54.853542 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:47:54.853921 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:54.854696 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:54.856056 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:54.856718 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:54.857077 (+ 359us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:54.857398 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:54.857421 (+ 23us) server_negotiation.cc:299] Negotiation successful
0807 08:47:54.857455 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":80,"threads_started":1}
I20260807 08:47:54.857733 10839 tablet_service.cc:1467] Processing CreateTablet for tablet b3e9220551594b1196716edefff00c4c (DEFAULT_TABLE table=test-table [id=140022e814af450f961e753adbe39b7d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:47:54.858095 10839 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3e9220551594b1196716edefff00c4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:47:54.858036 11028 tablet_service.cc:1467] Processing CreateTablet for tablet b3e9220551594b1196716edefff00c4c (DEFAULT_TABLE table=test-table [id=140022e814af450f961e753adbe39b7d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:47:54.858217 11225 tablet_service.cc:1467] Processing CreateTablet for tablet b3e9220551594b1196716edefff00c4c (DEFAULT_TABLE table=test-table [id=140022e814af450f961e753adbe39b7d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:47:54.858372 11028 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3e9220551594b1196716edefff00c4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:47:54.858532 11225 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3e9220551594b1196716edefff00c4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:47:54.860630 11353 tablet_bootstrap.cc:492] T b3e9220551594b1196716edefff00c4c P 0849bc73ccd44fca824ee36227576769: Bootstrap starting.
I20260807 08:47:54.861275 11353 tablet_bootstrap.cc:654] T b3e9220551594b1196716edefff00c4c P 0849bc73ccd44fca824ee36227576769: Neither blocks nor log segments found. Creating new log.
I20260807 08:47:54.861547 11355 tablet_bootstrap.cc:492] T b3e9220551594b1196716edefff00c4c P cd382bda5f714af5802c6308f570fb4e: Bootstrap starting.
I20260807 08:47:54.861927 11353 log.cc:824] T b3e9220551594b1196716edefff00c4c P 0849bc73ccd44fca824ee36227576769: Log is configured to *not* fsync() on all Append() calls
I20260807 08:47:54.862174 11355 tablet_bootstrap.cc:654] T b3e9220551594b1196716edefff00c4c P cd382bda5f714af5802c6308f570fb4e: Neither blocks nor log segments found. Creating new log.
I20260807 08:47:54.862478 11356 tablet_bootstrap.cc:492] T b3e9220551594b1196716edefff00c4c P aa5a9b62776a40f7ac603cb55c978167: Bootstrap starting.
I20260807 08:47:54.862673 11353 tablet_bootstrap.cc:492] T b3e9220551594b1196716edefff00c4c P 0849bc73ccd44fca824ee36227576769: No bootstrap required, opened a new log
I20260807 08:47:54.862744 11353 ts_tablet_manager.cc:1397] T b3e9220551594b1196716edefff00c4c P 0849bc73ccd44fca824ee36227576769: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 08:47:54.862748 11355 log.cc:824] T b3e9220551594b1196716edefff00c4c P cd382bda5f714af5802c6308f570fb4e: Log is configured to *not* fsync() on all Append() calls
I20260807 08:47:54.863042 11356 tablet_bootstrap.cc:654] T b3e9220551594b1196716edefff00c4c P aa5a9b62776a40f7ac603cb55c978167: Neither blocks nor log segments found. Creating new log.
I20260807 08:47:54.863533 11356 log.cc:824] T b3e9220551594b1196716edefff00c4c P aa5a9b62776a40f7ac603cb55c978167: Log is configured to *not* fsync() on all Append() calls
I20260807 08:47:54.863543 11355 tablet_bootstrap.cc:492] T b3e9220551594b1196716edefff00c4c P cd382bda5f714af5802c6308f570fb4e: No bootstrap required, opened a new log
I20260807 08:47:54.863616 11355 ts_tablet_manager.cc:1397] T b3e9220551594b1196716edefff00c4c P cd382bda5f714af5802c6308f570fb4e: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260807 08:47:54.864144 11356 tablet_bootstrap.cc:492] T b3e9220551594b1196716edefff00c4c P aa5a9b62776a40f7ac603cb55c978167: No bootstrap required, opened a new log
I20260807 08:47:54.864203 11356 ts_tablet_manager.cc:1397] T b3e9220551594b1196716edefff00c4c P aa5a9b62776a40f7ac603cb55c978167: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 08:47:54.864535 11353 raft_consensus.cc:348] T b3e9220551594b1196716edefff00c4c P 0849bc73ccd44fca824ee36227576769 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0849bc73ccd44fca824ee36227576769" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 41539 } } peers { permanent_uuid: "aa5a9b62776a40f7ac603cb55c978167" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 44231 } } peers { permanent_uuid: "cd382bda5f714af5802c6308f570fb4e" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 38293 } }
I20260807 08:47:54.864710 11353 raft_consensus.cc:374] T b3e9220551594b1196716edefff00c4c P 0849bc73ccd44fca824ee36227576769 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:47:54.864778 11353 raft_consensus.cc:729] T b3e9220551594b1196716edefff00c4c P 0849bc73ccd44fca824ee36227576769 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0849bc73ccd44fca824ee36227576769, State: Initialized, Role: FOLLOWER
I20260807 08:47:54.864914 11353 consensus_queue.cc:260] T b3e9220551594b1196716edefff00c4c P 0849bc73ccd44fca824ee36227576769 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0849bc73ccd44fca824ee36227576769" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 41539 } } peers { permanent_uuid: "aa5a9b62776a40f7ac603cb55c978167" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 44231 } } peers { permanent_uuid: "cd382bda5f714af5802c6308f570fb4e" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 38293 } }
I20260807 08:47:54.865195 11332 heartbeater.cc:503] Master 127.4.159.190:37935 was elected leader, sending a full tablet report...
I20260807 08:47:54.865427 11353 ts_tablet_manager.cc:1428] T b3e9220551594b1196716edefff00c4c P 0849bc73ccd44fca824ee36227576769: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 08:47:54.865784 11355 raft_consensus.cc:348] T b3e9220551594b1196716edefff00c4c P cd382bda5f714af5802c6308f570fb4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0849bc73ccd44fca824ee36227576769" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 41539 } } peers { permanent_uuid: "aa5a9b62776a40f7ac603cb55c978167" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 44231 } } peers { permanent_uuid: "cd382bda5f714af5802c6308f570fb4e" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 38293 } }
I20260807 08:47:54.865906 11355 raft_consensus.cc:374] T b3e9220551594b1196716edefff00c4c P cd382bda5f714af5802c6308f570fb4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:47:54.865936 11355 raft_consensus.cc:729] T b3e9220551594b1196716edefff00c4c P cd382bda5f714af5802c6308f570fb4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd382bda5f714af5802c6308f570fb4e, State: Initialized, Role: FOLLOWER
I20260807 08:47:54.865923 11356 raft_consensus.cc:348] T b3e9220551594b1196716edefff00c4c P aa5a9b62776a40f7ac603cb55c978167 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0849bc73ccd44fca824ee36227576769" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 41539 } } peers { permanent_uuid: "aa5a9b62776a40f7ac603cb55c978167" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 44231 } } peers { permanent_uuid: "cd382bda5f714af5802c6308f570fb4e" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 38293 } }
I20260807 08:47:54.866077 11355 consensus_queue.cc:260] T b3e9220551594b1196716edefff00c4c P cd382bda5f714af5802c6308f570fb4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0849bc73ccd44fca824ee36227576769" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 41539 } } peers { permanent_uuid: "aa5a9b62776a40f7ac603cb55c978167" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 44231 } } peers { permanent_uuid: "cd382bda5f714af5802c6308f570fb4e" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 38293 } }
I20260807 08:47:54.866135 11356 raft_consensus.cc:374] T b3e9220551594b1196716edefff00c4c P aa5a9b62776a40f7ac603cb55c978167 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:47:54.866179 11356 raft_consensus.cc:729] T b3e9220551594b1196716edefff00c4c P aa5a9b62776a40f7ac603cb55c978167 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa5a9b62776a40f7ac603cb55c978167, State: Initialized, Role: FOLLOWER
I20260807 08:47:54.866307 11356 consensus_queue.cc:260] T b3e9220551594b1196716edefff00c4c P aa5a9b62776a40f7ac603cb55c978167 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0849bc73ccd44fca824ee36227576769" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 41539 } } peers { permanent_uuid: "aa5a9b62776a40f7ac603cb55c978167" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 44231 } } peers { permanent_uuid: "cd382bda5f714af5802c6308f570fb4e" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 38293 } }
I20260807 08:47:54.866422 11355 ts_tablet_manager.cc:1428] T b3e9220551594b1196716edefff00c4c P cd382bda5f714af5802c6308f570fb4e: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260807 08:47:54.866546 11125 heartbeater.cc:503] Master 127.4.159.190:37935 was elected leader, sending a full tablet report...
I20260807 08:47:54.866608 10938 heartbeater.cc:503] Master 127.4.159.190:37935 was elected leader, sending a full tablet report...
I20260807 08:47:54.866874 11356 ts_tablet_manager.cc:1428] T b3e9220551594b1196716edefff00c4c P aa5a9b62776a40f7ac603cb55c978167: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260807 08:47:54.869563 11364 raft_consensus.cc:482] T b3e9220551594b1196716edefff00c4c P aa5a9b62776a40f7ac603cb55c978167 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:47:54.869644 11364 raft_consensus.cc:504] T b3e9220551594b1196716edefff00c4c P aa5a9b62776a40f7ac603cb55c978167 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0849bc73ccd44fca824ee36227576769" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 41539 } } peers { permanent_uuid: "aa5a9b62776a40f7ac603cb55c978167" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 44231 } } peers { permanent_uuid: "cd382bda5f714af5802c6308f570fb4e" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 38293 } }
I20260807 08:47:54.869889 11364 leader_election.cc:288] T b3e9220551594b1196716edefff00c4c P aa5a9b62776a40f7ac603cb55c978167 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0849bc73ccd44fca824ee36227576769 (127.4.159.131:41539), cd382bda5f714af5802c6308f570fb4e (127.4.159.130:38293)
I20260807 08:47:54.872399 11348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:54.870058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.129:55531 (local address 127.4.159.131:41539)
0807 08:47:54.870112 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:54.870113 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:54.870235 (+ 122us) server_negotiation.cc:407] Connection header received
0807 08:47:54.870255 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:54.870256 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:54.870263 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:54.870312 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:47:54.870462 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:54.870913 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:54.871760 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:54.872120 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:54.872176 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:54.872183 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:54.872316 (+ 133us) server_negotiation.cc:299] Negotiation successful
0807 08:47:54.872345 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260807 08:47:54.872470 11351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:54.870184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.129:35475 (local address 127.4.159.130:38293)
0807 08:47:54.870220 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:54.870222 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:54.870274 (+ 52us) server_negotiation.cc:407] Connection header received
0807 08:47:54.870322 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:54.870322 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:54.870331 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:54.870418 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:47:54.870538 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:54.870977 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:54.871829 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:54.872183 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:54.872240 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:54.872247 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:54.872382 (+ 135us) server_negotiation.cc:299] Negotiation successful
0807 08:47:54.872411 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 08:47:54.872495 11365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:54.870103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.130:38293 (local address 127.4.159.129:35475)
0807 08:47:54.870224 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0807 08:47:54.870233 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 08:47:54.870278 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:47:54.870422 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:47:54.870423 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:47:54.870424 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:47:54.870483 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:54.870485 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:54.871039 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:54.871040 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:54.871756 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:54.871758 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:54.872248 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:54.872249 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:54.872325 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:54.872328 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 08:47:54.872372 (+ 44us) client_negotiation.cc:240] Negotiation successful
0807 08:47:54.872413 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":78,"threads_started":1}
I20260807 08:47:54.872402 10940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:54.870005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.131:41539 (local address 127.4.159.129:55531)
0807 08:47:54.870098 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0807 08:47:54.870108 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 08:47:54.870162 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:47:54.870325 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:47:54.870326 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:47:54.870328 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:47:54.870406 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:54.870408 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:54.870973 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:54.870974 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:54.871685 (+ 711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:54.871687 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:54.872185 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:54.872185 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:54.872260 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:54.872263 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 08:47:54.872306 (+ 43us) client_negotiation.cc:240] Negotiation successful
0807 08:47:54.872339 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":75}
I20260807 08:47:54.872983 11056 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3e9220551594b1196716edefff00c4c" candidate_uuid: "aa5a9b62776a40f7ac603cb55c978167" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd382bda5f714af5802c6308f570fb4e" is_pre_election: true
I20260807 08:47:54.873047 11260 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3e9220551594b1196716edefff00c4c" candidate_uuid: "aa5a9b62776a40f7ac603cb55c978167" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0849bc73ccd44fca824ee36227576769" is_pre_election: true
I20260807 08:47:54.873091 11056 raft_consensus.cc:2454] T b3e9220551594b1196716edefff00c4c P cd382bda5f714af5802c6308f570fb4e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa5a9b62776a40f7ac603cb55c978167 in term 0.
I20260807 08:47:54.873167 11260 raft_consensus.cc:2454] T b3e9220551594b1196716edefff00c4c P 0849bc73ccd44fca824ee36227576769 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa5a9b62776a40f7ac603cb55c978167 in term 0.
I20260807 08:47:54.873291 10774 leader_election.cc:302] T b3e9220551594b1196716edefff00c4c P aa5a9b62776a40f7ac603cb55c978167 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aa5a9b62776a40f7ac603cb55c978167, cd382bda5f714af5802c6308f570fb4e; no voters:
I20260807 08:47:54.873389 11364 raft_consensus.cc:2781] T b3e9220551594b1196716edefff00c4c P aa5a9b62776a40f7ac603cb55c978167 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:47:54.873440 11364 raft_consensus.cc:482] T b3e9220551594b1196716edefff00c4c P aa5a9b62776a40f7ac603cb55c978167 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:47:54.873468 11364 raft_consensus.cc:3037] T b3e9220551594b1196716edefff00c4c P aa5a9b62776a40f7ac603cb55c978167 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:47:54.874078 11364 raft_consensus.cc:504] T b3e9220551594b1196716edefff00c4c P aa5a9b62776a40f7ac603cb55c978167 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0849bc73ccd44fca824ee36227576769" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 41539 } } peers { permanent_uuid: "aa5a9b62776a40f7ac603cb55c978167" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 44231 } } peers { permanent_uuid: "cd382bda5f714af5802c6308f570fb4e" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 38293 } }
I20260807 08:47:54.874178 11364 leader_election.cc:288] T b3e9220551594b1196716edefff00c4c P aa5a9b62776a40f7ac603cb55c978167 [CANDIDATE]: Term 1 election: Requested vote from peers 0849bc73ccd44fca824ee36227576769 (127.4.159.131:41539), cd382bda5f714af5802c6308f570fb4e (127.4.159.130:38293)
I20260807 08:47:54.874333 11260 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3e9220551594b1196716edefff00c4c" candidate_uuid: "aa5a9b62776a40f7ac603cb55c978167" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0849bc73ccd44fca824ee36227576769"
I20260807 08:47:54.874362 11056 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3e9220551594b1196716edefff00c4c" candidate_uuid: "aa5a9b62776a40f7ac603cb55c978167" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd382bda5f714af5802c6308f570fb4e"
I20260807 08:47:54.874423 11260 raft_consensus.cc:3037] T b3e9220551594b1196716edefff00c4c P 0849bc73ccd44fca824ee36227576769 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:47:54.874423 11056 raft_consensus.cc:3037] T b3e9220551594b1196716edefff00c4c P cd382bda5f714af5802c6308f570fb4e [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:47:54.874979 11260 raft_consensus.cc:2454] T b3e9220551594b1196716edefff00c4c P 0849bc73ccd44fca824ee36227576769 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa5a9b62776a40f7ac603cb55c978167 in term 1.
I20260807 08:47:54.874984 11056 raft_consensus.cc:2454] T b3e9220551594b1196716edefff00c4c P cd382bda5f714af5802c6308f570fb4e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa5a9b62776a40f7ac603cb55c978167 in term 1.
I20260807 08:47:54.875121 10774 leader_election.cc:302] T b3e9220551594b1196716edefff00c4c P aa5a9b62776a40f7ac603cb55c978167 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0849bc73ccd44fca824ee36227576769, aa5a9b62776a40f7ac603cb55c978167; no voters:
I20260807 08:47:54.875216 11364 raft_consensus.cc:2781] T b3e9220551594b1196716edefff00c4c P aa5a9b62776a40f7ac603cb55c978167 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:47:54.875353 11364 raft_consensus.cc:686] T b3e9220551594b1196716edefff00c4c P aa5a9b62776a40f7ac603cb55c978167 [term 1 LEADER]: Becoming Leader. State: Replica: aa5a9b62776a40f7ac603cb55c978167, State: Running, Role: LEADER
I20260807 08:47:54.875430 11364 consensus_queue.cc:237] T b3e9220551594b1196716edefff00c4c P aa5a9b62776a40f7ac603cb55c978167 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0849bc73ccd44fca824ee36227576769" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 41539 } } peers { permanent_uuid: "aa5a9b62776a40f7ac603cb55c978167" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 44231 } } peers { permanent_uuid: "cd382bda5f714af5802c6308f570fb4e" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 38293 } }
I20260807 08:47:54.876128 10665 catalog_manager.cc:5495] T b3e9220551594b1196716edefff00c4c P aa5a9b62776a40f7ac603cb55c978167 reported cstate change: term changed from 0 to 1, leader changed from <none> to aa5a9b62776a40f7ac603cb55c978167 (127.4.159.129). New cstate: current_term: 1 leader_uuid: "aa5a9b62776a40f7ac603cb55c978167" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0849bc73ccd44fca824ee36227576769" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 41539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa5a9b62776a40f7ac603cb55c978167" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 44231 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd382bda5f714af5802c6308f570fb4e" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 38293 } health_report { overall_health: UNKNOWN } } }
I20260807 08:47:54.883354 11349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:54.881282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55638 (local address 127.4.159.129:44231)
0807 08:47:54.881316 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:54.881318 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:54.881325 (+ 7us) server_negotiation.cc:407] Connection header received
0807 08:47:54.881372 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:54.881373 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:54.881388 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:54.881448 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:47:54.881557 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:54.881970 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:54.882712 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:54.882895 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:54.882942 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:54.883075 (+ 133us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:47:54.883161 (+ 86us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:47:54.883227 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:54.883263 (+ 36us) server_negotiation.cc:299] Negotiation successful
0807 08:47:54.883280 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260807 08:47:54.884656 11056 raft_consensus.cc:1261] T b3e9220551594b1196716edefff00c4c P cd382bda5f714af5802c6308f570fb4e [term 1 FOLLOWER]: Refusing update from remote peer aa5a9b62776a40f7ac603cb55c978167: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:47:54.884672 11260 raft_consensus.cc:1261] T b3e9220551594b1196716edefff00c4c P 0849bc73ccd44fca824ee36227576769 [term 1 FOLLOWER]: Refusing update from remote peer aa5a9b62776a40f7ac603cb55c978167: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:47:54.884922 11364 consensus_queue.cc:1035] T b3e9220551594b1196716edefff00c4c P aa5a9b62776a40f7ac603cb55c978167 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0849bc73ccd44fca824ee36227576769" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 41539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:47:54.885041 11366 consensus_queue.cc:1035] T b3e9220551594b1196716edefff00c4c P aa5a9b62776a40f7ac603cb55c978167 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd382bda5f714af5802c6308f570fb4e" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 38293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:47:54.889181 10665 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50734:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:47:54.889248 10665 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50734:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:47:54.889844 10665 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6194428e451d438ebd95dc7e042dacc9: Sending DeleteTablet for 3 replicas of tablet b3e9220551594b1196716edefff00c4c
I20260807 08:47:54.890144 10839 tablet_service.cc:1514] Processing DeleteTablet for tablet b3e9220551594b1196716edefff00c4c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:47:54 UTC) from {username='kudu', principal='kudu/127.4.159.190@KRBTEST.COM'} at 127.0.0.1:55632
I20260807 08:47:54.890177 11225 tablet_service.cc:1514] Processing DeleteTablet for tablet b3e9220551594b1196716edefff00c4c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:47:54 UTC) from {username='kudu', principal='kudu/127.4.159.190@KRBTEST.COM'} at 127.0.0.1:48416
I20260807 08:47:54.890367 11028 tablet_service.cc:1514] Processing DeleteTablet for tablet b3e9220551594b1196716edefff00c4c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:47:54 UTC) from {username='kudu', principal='kudu/127.4.159.190@KRBTEST.COM'} at 127.0.0.1:42188
I20260807 08:47:54.890534 11388 tablet_replica.cc:330] T b3e9220551594b1196716edefff00c4c P aa5a9b62776a40f7ac603cb55c978167: stopping tablet replica
I20260807 08:47:54.890635 11388 raft_consensus.cc:2229] T b3e9220551594b1196716edefff00c4c P aa5a9b62776a40f7ac603cb55c978167 [term 1 LEADER]: Raft consensus shutting down.
I20260807 08:47:54.890646 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 10748
I20260807 08:47:54.890743 11388 raft_consensus.cc:2258] T b3e9220551594b1196716edefff00c4c P aa5a9b62776a40f7ac603cb55c978167 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:47:54.890865 11391 tablet_replica.cc:330] T b3e9220551594b1196716edefff00c4c P 0849bc73ccd44fca824ee36227576769: stopping tablet replica
I20260807 08:47:54.890928 11391 raft_consensus.cc:2229] T b3e9220551594b1196716edefff00c4c P 0849bc73ccd44fca824ee36227576769 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:47:54.890990 11391 raft_consensus.cc:2258] T b3e9220551594b1196716edefff00c4c P 0849bc73ccd44fca824ee36227576769 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:47:54.891321 11392 tablet_replica.cc:330] T b3e9220551594b1196716edefff00c4c P cd382bda5f714af5802c6308f570fb4e: stopping tablet replica
I20260807 08:47:54.891424 11392 raft_consensus.cc:2229] T b3e9220551594b1196716edefff00c4c P cd382bda5f714af5802c6308f570fb4e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:47:54.891530 11392 raft_consensus.cc:2258] T b3e9220551594b1196716edefff00c4c P cd382bda5f714af5802c6308f570fb4e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:47:54.893366 11392 ts_tablet_manager.cc:1905] T b3e9220551594b1196716edefff00c4c P cd382bda5f714af5802c6308f570fb4e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:47:54.895243 11392 ts_tablet_manager.cc:1918] T b3e9220551594b1196716edefff00c4c P cd382bda5f714af5802c6308f570fb4e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:47:54.895305 11392 log.cc:1196] T b3e9220551594b1196716edefff00c4c P cd382bda5f714af5802c6308f570fb4e: Deleting WAL directory at /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786092453486433-4734-0/minicluster-data/ts-1/wal/wals/b3e9220551594b1196716edefff00c4c
I20260807 08:47:54.895551 11392 ts_tablet_manager.cc:1939] T b3e9220551594b1196716edefff00c4c P cd382bda5f714af5802c6308f570fb4e: Deleting consensus metadata
I20260807 08:47:54.895885 10645 catalog_manager.cc:4841] TS cd382bda5f714af5802c6308f570fb4e (127.4.159.130:38293): tablet b3e9220551594b1196716edefff00c4c (table test-table [id=140022e814af450f961e753adbe39b7d]) successfully deleted
I20260807 08:47:54.896696 11391 ts_tablet_manager.cc:1905] T b3e9220551594b1196716edefff00c4c P 0849bc73ccd44fca824ee36227576769: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:47:54.897993 11391 ts_tablet_manager.cc:1918] T b3e9220551594b1196716edefff00c4c P 0849bc73ccd44fca824ee36227576769: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:47:54.898061 11391 log.cc:1196] T b3e9220551594b1196716edefff00c4c P 0849bc73ccd44fca824ee36227576769: Deleting WAL directory at /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786092453486433-4734-0/minicluster-data/ts-2/wal/wals/b3e9220551594b1196716edefff00c4c
I20260807 08:47:54.898283 11391 ts_tablet_manager.cc:1939] T b3e9220551594b1196716edefff00c4c P 0849bc73ccd44fca824ee36227576769: Deleting consensus metadata
I20260807 08:47:54.898679 10645 catalog_manager.cc:4841] TS 0849bc73ccd44fca824ee36227576769 (127.4.159.131:41539): tablet b3e9220551594b1196716edefff00c4c (table test-table [id=140022e814af450f961e753adbe39b7d]) successfully deleted
W20260807 08:47:54.900794 10644 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.159.129:44231 (error 108)
I20260807 08:47:54.901283 11346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:54.901163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.129:44231 (local address 127.0.0.1:55654)
0807 08:47:54.901212 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0807 08:47:54.901253 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.129:44231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260807 08:47:54.901288 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 10944
W20260807 08:47:54.902685 10644 catalog_manager.cc:4584] TS aa5a9b62776a40f7ac603cb55c978167 (127.4.159.129:44231): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b3e9220551594b1196716edefff00c4c: Network error: Client connection negotiation failed: client connection to 127.4.159.129:44231: connect: Connection refused (error 111)
I20260807 08:47:54.911712 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 11137
I20260807 08:47:54.925885 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 10617
2026-08-07T08:47:54Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (1908 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-07T08:47:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:47:54Z Disabled control of system clock
I20260807 08:47:54.953985 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786092453486433-4734-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786092453486433-4734-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786092453486433-4734-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.159.190:33919
--webserver_interface=127.4.159.190
--webserver_port=0
--builtin_ntp_servers=127.4.159.148:39471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.159.190:33919
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260807 08:47:55.031038 11398 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:47:55.031209 11398 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:47:55.031227 11398 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:47:55.032250 11398 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:47:55.032280 11398 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:47:55.032291 11398 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:47:55.032302 11398 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:47:55.032315 11398 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:47:55.033535 11398 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:39471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786092453486433-4734-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786092453486433-4734-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.159.190:33919
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.159.190:33919
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.159.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11398
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786092453486433-4734-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:47:55.033718 11398 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:47:55.033931 11398 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:47:55.038142 11414 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:47:55.038153 11412 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:47:55.038211 11411 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:47:55.038332 11398 server_base.cc:1034] running on GCE node
I20260807 08:47:55.038662 11398 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:47:55.038910 11398 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:47:55.040035 11398 hybrid_clock.cc:648] HybridClock initialized: now 1786092475040027 us; error 27 us; skew 500 ppm
I20260807 08:47:55.041134 11398 webserver.cc:460] Webserver started at http://127.4.159.190:40219/ using document root <none> and password file <none>
I20260807 08:47:55.041334 11398 fs_manager.cc:359] Metadata directory not provided
I20260807 08:47:55.041381 11398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:47:55.041497 11398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:47:55.042457 11398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786092453486433-4734-0/minicluster-data/master-0/data/instance:
uuid: "03c2eb4951344907a5f727408b58ac4c"
format_stamp: "Formatted at 2026-08-07 08:47:55 on dist-test-slave-6zbq"
server_key: "26272b88db61b5cf35f20b486512032d"
server_key_iv: "379f7916e76acf293646f154480bc7cd"
server_key_version: "encryptionkey@0"
I20260807 08:47:55.042761 11398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786092453486433-4734-0/minicluster-data/master-0/wal/instance:
uuid: "03c2eb4951344907a5f727408b58ac4c"
format_stamp: "Formatted at 2026-08-07 08:47:55 on dist-test-slave-6zbq"
server_key: "26272b88db61b5cf35f20b486512032d"
server_key_iv: "379f7916e76acf293646f154480bc7cd"
server_key_version: "encryptionkey@0"
I20260807 08:47:55.043892 11398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:47:55.044539 11422 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:55.044678 11398 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 08:47:55.044756 11398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786092453486433-4734-0/minicluster-data/master-0/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786092453486433-4734-0/minicluster-data/master-0/wal
uuid: "03c2eb4951344907a5f727408b58ac4c"
format_stamp: "Formatted at 2026-08-07 08:47:55 on dist-test-slave-6zbq"
server_key: "26272b88db61b5cf35f20b486512032d"
server_key_iv: "379f7916e76acf293646f154480bc7cd"
server_key_version: "encryptionkey@0"
I20260807 08:47:55.044812 11398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786092453486433-4734-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786092453486433-4734-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786092453486433-4734-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:47:55.047901 11398 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:47:55.048135 11398 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:47:55.048249 11398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:47:55.051780 11398 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.190:33919
I20260807 08:47:55.051812 11494 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.190:33919 every 8 connection(s)
I20260807 08:47:55.052078 11398 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
I20260807 08:47:55.052654 11496 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:47:55.055222 11496 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03c2eb4951344907a5f727408b58ac4c: Bootstrap starting.
I20260807 08:47:55.055724 11496 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 03c2eb4951344907a5f727408b58ac4c: Neither blocks nor log segments found. Creating new log.
I20260807 08:47:55.055950 11496 log.cc:824] T 00000000000000000000000000000000 P 03c2eb4951344907a5f727408b58ac4c: Log is configured to *not* fsync() on all Append() calls
I20260807 08:47:55.056500 11496 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03c2eb4951344907a5f727408b58ac4c: No bootstrap required, opened a new log
I20260807 08:47:55.057679 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 11398
I20260807 08:47:55.057770 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786092453486433-4734-0/minicluster-data/master-0/wal/instance
I20260807 08:47:55.057919 4734 external_mini_cluster.cc:1428] Setting key 0c0d01a2f14b9fe51fd821624f382907
I20260807 08:47:55.058053 11496 raft_consensus.cc:348] T 00000000000000000000000000000000 P 03c2eb4951344907a5f727408b58ac4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03c2eb4951344907a5f727408b58ac4c" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 33919 } }
I20260807 08:47:55.058238 11496 raft_consensus.cc:374] T 00000000000000000000000000000000 P 03c2eb4951344907a5f727408b58ac4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:47:55.058279 11496 raft_consensus.cc:729] T 00000000000000000000000000000000 P 03c2eb4951344907a5f727408b58ac4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03c2eb4951344907a5f727408b58ac4c, State: Initialized, Role: FOLLOWER
I20260807 08:47:55.058424 11496 consensus_queue.cc:260] T 00000000000000000000000000000000 P 03c2eb4951344907a5f727408b58ac4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03c2eb4951344907a5f727408b58ac4c" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 33919 } }
I20260807 08:47:55.058507 11496 raft_consensus.cc:388] T 00000000000000000000000000000000 P 03c2eb4951344907a5f727408b58ac4c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:47:55.058543 11496 raft_consensus.cc:482] T 00000000000000000000000000000000 P 03c2eb4951344907a5f727408b58ac4c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:47:55.058583 11496 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 03c2eb4951344907a5f727408b58ac4c [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:47:55.059048 11496 raft_consensus.cc:504] T 00000000000000000000000000000000 P 03c2eb4951344907a5f727408b58ac4c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03c2eb4951344907a5f727408b58ac4c" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 33919 } }
I20260807 08:47:55.059163 11496 leader_election.cc:302] T 00000000000000000000000000000000 P 03c2eb4951344907a5f727408b58ac4c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 03c2eb4951344907a5f727408b58ac4c; no voters:
I20260807 08:47:55.059317 11496 leader_election.cc:288] T 00000000000000000000000000000000 P 03c2eb4951344907a5f727408b58ac4c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:47:55.059342 11501 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 03c2eb4951344907a5f727408b58ac4c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:47:55.059481 11501 raft_consensus.cc:686] T 00000000000000000000000000000000 P 03c2eb4951344907a5f727408b58ac4c [term 1 LEADER]: Becoming Leader. State: Replica: 03c2eb4951344907a5f727408b58ac4c, State: Running, Role: LEADER
I20260807 08:47:55.059557 11496 sys_catalog.cc:564] T 00000000000000000000000000000000 P 03c2eb4951344907a5f727408b58ac4c [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:47:55.059571 11501 consensus_queue.cc:237] T 00000000000000000000000000000000 P 03c2eb4951344907a5f727408b58ac4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03c2eb4951344907a5f727408b58ac4c" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 33919 } }
I20260807 08:47:55.059888 11501 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03c2eb4951344907a5f727408b58ac4c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 03c2eb4951344907a5f727408b58ac4c. Latest consensus state: current_term: 1 leader_uuid: "03c2eb4951344907a5f727408b58ac4c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03c2eb4951344907a5f727408b58ac4c" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 33919 } } }
I20260807 08:47:55.059957 11501 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03c2eb4951344907a5f727408b58ac4c [sys.catalog]: This master's current role is: LEADER
I20260807 08:47:55.060189 11505 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:47:55.060302 11502 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03c2eb4951344907a5f727408b58ac4c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "03c2eb4951344907a5f727408b58ac4c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03c2eb4951344907a5f727408b58ac4c" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 33919 } } }
I20260807 08:47:55.060381 11502 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03c2eb4951344907a5f727408b58ac4c [sys.catalog]: This master's current role is: LEADER
I20260807 08:47:55.060495 11500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:55.058406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38228 (local address 127.4.159.190:33919)
0807 08:47:55.058624 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:55.058631 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:55.058661 (+ 30us) server_negotiation.cc:407] Connection header received
0807 08:47:55.058763 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:55.058767 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:55.058797 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:55.058903 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:47:55.059560 (+ 657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:47:55.059564 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:47:55.059564 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:47:55.059569 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:47:55.059610 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:47:55.059637 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:47:55.059831 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:55.059861 (+ 30us) server_negotiation.cc:299] Negotiation successful
0807 08:47:55.059912 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":83,"threads_started":1}
I20260807 08:47:55.060891 11505 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:47:55.062282 11505 catalog_manager.cc:1347] Generated new cluster ID: 02b68ace7a004e0f81ba2e08b58cab30
I20260807 08:47:55.062361 11505 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:47:55.077409 11505 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:47:55.077534 11505 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:47:55.080188 11505 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 03c2eb4951344907a5f727408b58ac4c: Generated new TSK 0
I20260807 08:47:55.080324 11505 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:47:55.086445 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786092453486433-4734-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786092453486433-4734-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786092453486433-4734-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786092453486433-4734-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.159.129:0
--local_ip_for_outbound_sockets=127.4.159.129
--webserver_interface=127.4.159.129
--webserver_port=0
--tserver_master_addrs=127.4.159.190:33919
--builtin_ntp_servers=127.4.159.148:39471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260807 08:47:55.165690 11531 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:47:55.165863 11531 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:47:55.165884 11531 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:47:55.167011 11531 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:47:55.167048 11531 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:47:55.167071 11531 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.159.129
I20260807 08:47:55.168355 11531 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:39471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786092453486433-4734-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786092453486433-4734-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.159.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786092453486433-4734-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.159.129
--webserver_port=0
--tserver_master_addrs=127.4.159.190:33919
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11531
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.159.129
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786092453486433-4734-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:47:55.168550 11531 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:47:55.168812 11531 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:47:55.173383 11538 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:47:55.173389 11537 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:47:55.173389 11540 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:47:55.173760 11531 server_base.cc:1034] running on GCE node
I20260807 08:47:55.173949 11531 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:47:55.174161 11531 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:47:55.175289 11531 hybrid_clock.cc:648] HybridClock initialized: now 1786092475175268 us; error 38 us; skew 500 ppm
I20260807 08:47:55.176373 11531 webserver.cc:460] Webserver started at http://127.4.159.129:40141/ using document root <none> and password file <none>
I20260807 08:47:55.176576 11531 fs_manager.cc:359] Metadata directory not provided
I20260807 08:47:55.176635 11531 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:47:55.176757 11531 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:47:55.177706 11531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786092453486433-4734-0/minicluster-data/ts-0/data/instance:
uuid: "012426fe79f34934829eae4bddb0e4f4"
format_stamp: "Formatted at 2026-08-07 08:47:55 on dist-test-slave-6zbq"
server_key: "3b7a05907f894e63c27514cca8c714bd"
server_key_iv: "ed5fe727da52936c0563084abd5c10f1"
server_key_version: "encryptionkey@0"
I20260807 08:47:55.178017 11531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786092453486433-4734-0/minicluster-data/ts-0/wal/instance:
uuid: "012426fe79f34934829eae4bddb0e4f4"
format_stamp: "Formatted at 2026-08-07 08:47:55 on dist-test-slave-6zbq"
server_key: "3b7a05907f894e63c27514cca8c714bd"
server_key_iv: "ed5fe727da52936c0563084abd5c10f1"
server_key_version: "encryptionkey@0"
I20260807 08:47:55.179243 11531 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 08:47:55.179857 11547 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:55.180012 11531 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 08:47:55.180090 11531 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786092453486433-4734-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786092453486433-4734-0/minicluster-data/ts-0/wal
uuid: "012426fe79f34934829eae4bddb0e4f4"
format_stamp: "Formatted at 2026-08-07 08:47:55 on dist-test-slave-6zbq"
server_key: "3b7a05907f894e63c27514cca8c714bd"
server_key_iv: "ed5fe727da52936c0563084abd5c10f1"
server_key_version: "encryptionkey@0"
I20260807 08:47:55.180152 11531 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786092453486433-4734-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786092453486433-4734-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786092453486433-4734-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:47:55.183295 11531 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:47:55.183512 11531 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:47:55.183631 11531 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:47:55.183804 11531 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:47:55.184062 11531 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:47:55.184092 11531 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:55.184116 11531 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:47:55.184134 11531 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:55.189268 11531 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.129:39787
I20260807 08:47:55.189334 11726 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.129:39787 every 8 connection(s)
I20260807 08:47:55.189571 11531 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786092453486433-4734-0/minicluster-data/ts-0/data/info.pb
I20260807 08:47:55.190968 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 11531
I20260807 08:47:55.191059 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786092453486433-4734-0/minicluster-data/ts-0/wal/instance
I20260807 08:47:55.191213 4734 external_mini_cluster.cc:1428] Setting key 11502fba55a36449e85f3ee682ed3e97
I20260807 08:47:55.192116 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786092453486433-4734-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786092453486433-4734-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786092453486433-4734-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786092453486433-4734-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.159.130:0
--local_ip_for_outbound_sockets=127.4.159.130
--webserver_interface=127.4.159.130
--webserver_port=0
--tserver_master_addrs=127.4.159.190:33919
--builtin_ntp_servers=127.4.159.148:39471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260807 08:47:55.192461 11500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:55.190162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.129:48235 (local address 127.4.159.190:33919)
0807 08:47:55.190215 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:55.190217 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:55.190437 (+ 220us) server_negotiation.cc:407] Connection header received
0807 08:47:55.190873 (+ 436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:55.190874 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:55.190882 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:55.190951 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:47:55.192191 (+ 1240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:47:55.192192 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:47:55.192193 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:47:55.192197 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:47:55.192217 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:47:55.192222 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:47:55.192279 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:55.192393 (+ 114us) server_negotiation.cc:299] Negotiation successful
0807 08:47:55.192408 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260807 08:47:55.192637 11732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:55.190152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:33919 (local address 127.4.159.129:48235)
0807 08:47:55.190306 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0807 08:47:55.190335 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0807 08:47:55.190766 (+ 431us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:47:55.190998 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:47:55.191001 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:47:55.191008 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:47:55.192028 (+ 1020us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 08:47:55.192028 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:47:55.192055 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 08:47:55.192083 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:47:55.192307 (+ 224us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:47:55.192315 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:47:55.192318 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 08:47:55.192389 (+ 71us) client_negotiation.cc:240] Negotiation successful
0807 08:47:55.192448 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":74,"threads_started":1}
I20260807 08:47:55.193998 11729 heartbeater.cc:347] Connected to a master server at 127.4.159.190:33919
I20260807 08:47:55.194077 11729 heartbeater.cc:464] Registering TS with master...
I20260807 08:47:55.194206 11729 heartbeater.cc:511] Master 127.4.159.190:33919 requested a full tablet report, sending...
I20260807 08:47:55.194665 11443 ts_manager.cc:194] Registered new tserver with Master: 012426fe79f34934829eae4bddb0e4f4 (127.4.159.129:39787)
W20260807 08:47:55.271229 11733 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:47:55.271405 11733 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:47:55.271435 11733 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:47:55.272529 11733 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:47:55.272564 11733 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:47:55.272598 11733 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.159.130
I20260807 08:47:55.273902 11733 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:39471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786092453486433-4734-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786092453486433-4734-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.159.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786092453486433-4734-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.159.130
--webserver_port=0
--tserver_master_addrs=127.4.159.190:33919
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11733
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.159.130
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786092453486433-4734-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:47:55.274120 11733 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:47:55.274411 11733 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:47:55.278879 11744 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:47:55.278936 11742 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:47:55.278895 11750 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:47:55.279206 11733 server_base.cc:1034] running on GCE node
I20260807 08:47:55.279350 11733 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:47:55.279533 11733 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:47:55.280670 11733 hybrid_clock.cc:648] HybridClock initialized: now 1786092475280655 us; error 28 us; skew 500 ppm
I20260807 08:47:55.281821 11733 webserver.cc:460] Webserver started at http://127.4.159.130:39111/ using document root <none> and password file <none>
I20260807 08:47:55.282029 11733 fs_manager.cc:359] Metadata directory not provided
I20260807 08:47:55.282073 11733 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:47:55.282202 11733 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:47:55.283136 11733 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786092453486433-4734-0/minicluster-data/ts-1/data/instance:
uuid: "0fc304540a354d048158cbb41c21cfd0"
format_stamp: "Formatted at 2026-08-07 08:47:55 on dist-test-slave-6zbq"
server_key: "c41c58a3bd370e03e886cd7ba7de83a1"
server_key_iv: "869bdead11187461ed44483e1a2eaa87"
server_key_version: "encryptionkey@0"
I20260807 08:47:55.283480 11733 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786092453486433-4734-0/minicluster-data/ts-1/wal/instance:
uuid: "0fc304540a354d048158cbb41c21cfd0"
format_stamp: "Formatted at 2026-08-07 08:47:55 on dist-test-slave-6zbq"
server_key: "c41c58a3bd370e03e886cd7ba7de83a1"
server_key_iv: "869bdead11187461ed44483e1a2eaa87"
server_key_version: "encryptionkey@0"
I20260807 08:47:55.284588 11733 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 08:47:55.285218 11759 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:55.285356 11733 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 08:47:55.285434 11733 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786092453486433-4734-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786092453486433-4734-0/minicluster-data/ts-1/wal
uuid: "0fc304540a354d048158cbb41c21cfd0"
format_stamp: "Formatted at 2026-08-07 08:47:55 on dist-test-slave-6zbq"
server_key: "c41c58a3bd370e03e886cd7ba7de83a1"
server_key_iv: "869bdead11187461ed44483e1a2eaa87"
server_key_version: "encryptionkey@0"
I20260807 08:47:55.285490 11733 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786092453486433-4734-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786092453486433-4734-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786092453486433-4734-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:47:55.288587 11733 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:47:55.288807 11733 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:47:55.288921 11733 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:47:55.289091 11733 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:47:55.289352 11733 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:47:55.289383 11733 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:55.289408 11733 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:47:55.289431 11733 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:55.294612 11733 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.130:37603
I20260807 08:47:55.294646 11923 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.130:37603 every 8 connection(s)
I20260807 08:47:55.294909 11733 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786092453486433-4734-0/minicluster-data/ts-1/data/info.pb
I20260807 08:47:55.297039 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 11733
I20260807 08:47:55.297124 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786092453486433-4734-0/minicluster-data/ts-1/wal/instance
I20260807 08:47:55.297317 4734 external_mini_cluster.cc:1428] Setting key ee367289971d2429c2ace7518df4a98b
I20260807 08:47:55.297458 11500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:55.295486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.130:42529 (local address 127.4.159.190:33919)
0807 08:47:55.295561 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:55.295563 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:55.295725 (+ 162us) server_negotiation.cc:407] Connection header received
0807 08:47:55.296112 (+ 387us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:55.296113 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:55.296118 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:55.296161 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:47:55.297215 (+ 1054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:47:55.297217 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:47:55.297217 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:47:55.297224 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:47:55.297245 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:47:55.297250 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:47:55.297317 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:55.297379 (+ 62us) server_negotiation.cc:299] Negotiation successful
0807 08:47:55.297393 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 08:47:55.297631 11929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:55.295464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:33919 (local address 127.4.159.130:42529)
0807 08:47:55.295626 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0807 08:47:55.295656 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0807 08:47:55.296026 (+ 370us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:47:55.296217 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:47:55.296220 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:47:55.296228 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:47:55.297036 (+ 808us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 08:47:55.297036 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:47:55.297061 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 08:47:55.297089 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:47:55.297316 (+ 227us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:47:55.297321 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:47:55.297323 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 08:47:55.297380 (+ 57us) client_negotiation.cc:240] Negotiation successful
0807 08:47:55.297439 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":74,"threads_started":1}
I20260807 08:47:55.298090 11924 heartbeater.cc:347] Connected to a master server at 127.4.159.190:33919
I20260807 08:47:55.298173 11924 heartbeater.cc:464] Registering TS with master...
I20260807 08:47:55.298290 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786092453486433-4734-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786092453486433-4734-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786092453486433-4734-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786092453486433-4734-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.159.131:0
--local_ip_for_outbound_sockets=127.4.159.131
--webserver_interface=127.4.159.131
--webserver_port=0
--tserver_master_addrs=127.4.159.190:33919
--builtin_ntp_servers=127.4.159.148:39471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260807 08:47:55.298305 11924 heartbeater.cc:511] Master 127.4.159.190:33919 requested a full tablet report, sending...
I20260807 08:47:55.298674 11442 ts_manager.cc:194] Registered new tserver with Master: 0fc304540a354d048158cbb41c21cfd0 (127.4.159.130:37603)
W20260807 08:47:55.377651 11930 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:47:55.377831 11930 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:47:55.377859 11930 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:47:55.378981 11930 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:47:55.379019 11930 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:47:55.379052 11930 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.159.131
I20260807 08:47:55.380349 11930 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:39471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786092453486433-4734-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786092453486433-4734-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.159.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786092453486433-4734-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.159.131
--webserver_port=0
--tserver_master_addrs=127.4.159.190:33919
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11930
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.159.131
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786092453486433-4734-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:47:55.380571 11930 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:47:55.380827 11930 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:47:55.385428 11938 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:47:55.385447 11930 server_base.cc:1034] running on GCE node
W20260807 08:47:55.385445 11939 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:47:55.385432 11943 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:47:55.385838 11930 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:47:55.386067 11930 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:47:55.387205 11930 hybrid_clock.cc:648] HybridClock initialized: now 1786092475387194 us; error 23 us; skew 500 ppm
I20260807 08:47:55.388345 11930 webserver.cc:460] Webserver started at http://127.4.159.131:36203/ using document root <none> and password file <none>
I20260807 08:47:55.388546 11930 fs_manager.cc:359] Metadata directory not provided
I20260807 08:47:55.388607 11930 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:47:55.388736 11930 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:47:55.389712 11930 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786092453486433-4734-0/minicluster-data/ts-2/data/instance:
uuid: "a15b209594fc406896b6461c06f4cd28"
format_stamp: "Formatted at 2026-08-07 08:47:55 on dist-test-slave-6zbq"
server_key: "6b5ecf7e241b022be0f7adddb4745ab3"
server_key_iv: "13d2f0c42b2f242552b666629dd22339"
server_key_version: "encryptionkey@0"
I20260807 08:47:55.390019 11930 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786092453486433-4734-0/minicluster-data/ts-2/wal/instance:
uuid: "a15b209594fc406896b6461c06f4cd28"
format_stamp: "Formatted at 2026-08-07 08:47:55 on dist-test-slave-6zbq"
server_key: "6b5ecf7e241b022be0f7adddb4745ab3"
server_key_iv: "13d2f0c42b2f242552b666629dd22339"
server_key_version: "encryptionkey@0"
I20260807 08:47:55.391242 11930 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 08:47:55.391878 11953 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:55.392025 11930 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 08:47:55.392102 11930 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786092453486433-4734-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786092453486433-4734-0/minicluster-data/ts-2/wal
uuid: "a15b209594fc406896b6461c06f4cd28"
format_stamp: "Formatted at 2026-08-07 08:47:55 on dist-test-slave-6zbq"
server_key: "6b5ecf7e241b022be0f7adddb4745ab3"
server_key_iv: "13d2f0c42b2f242552b666629dd22339"
server_key_version: "encryptionkey@0"
I20260807 08:47:55.392153 11930 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786092453486433-4734-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786092453486433-4734-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786092453486433-4734-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:47:55.395179 11930 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:47:55.395364 11930 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:47:55.395450 11930 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:47:55.395583 11930 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:47:55.395808 11930 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:47:55.395843 11930 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:55.395860 11930 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:47:55.395877 11930 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:55.401110 11930 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.131:32987
I20260807 08:47:55.401173 12116 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.131:32987 every 8 connection(s)
I20260807 08:47:55.401400 11930 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786092453486433-4734-0/minicluster-data/ts-2/data/info.pb
I20260807 08:47:55.402920 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 11930
I20260807 08:47:55.402995 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786092453486433-4734-0/minicluster-data/ts-2/wal/instance
I20260807 08:47:55.403175 4734 external_mini_cluster.cc:1428] Setting key 4174e5540e312801cadd87f79e5e7099
I20260807 08:47:55.404052 11500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:55.401992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.131:57797 (local address 127.4.159.190:33919)
0807 08:47:55.402044 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:55.402046 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:55.402230 (+ 184us) server_negotiation.cc:407] Connection header received
0807 08:47:55.402679 (+ 449us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:55.402680 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:55.402687 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:55.402738 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:47:55.403838 (+ 1100us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:47:55.403839 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:47:55.403839 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:47:55.403842 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:47:55.403859 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:47:55.403863 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:47:55.403911 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:55.403973 (+ 62us) server_negotiation.cc:299] Negotiation successful
0807 08:47:55.403986 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260807 08:47:55.404242 12121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:55.401974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:33919 (local address 127.4.159.131:57797)
0807 08:47:55.402133 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0807 08:47:55.402160 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0807 08:47:55.402587 (+ 427us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:47:55.402822 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:47:55.402826 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:47:55.402835 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:47:55.403703 (+ 868us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 08:47:55.403703 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:47:55.403729 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 08:47:55.403763 (+ 34us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:47:55.403910 (+ 147us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:47:55.403914 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:47:55.403916 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 08:47:55.403982 (+ 66us) client_negotiation.cc:240] Negotiation successful
0807 08:47:55.404039 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":63,"threads_started":1}
I20260807 08:47:55.404633 12117 heartbeater.cc:347] Connected to a master server at 127.4.159.190:33919
I20260807 08:47:55.404704 12117 heartbeater.cc:464] Registering TS with master...
I20260807 08:47:55.404834 12117 heartbeater.cc:511] Master 127.4.159.190:33919 requested a full tablet report, sending...
I20260807 08:47:55.405129 11442 ts_manager.cc:194] Registered new tserver with Master: a15b209594fc406896b6461c06f4cd28 (127.4.159.131:32987)
I20260807 08:47:55.414673 4734 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 08:47:55.416664 11500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:55.415839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38234 (local address 127.4.159.190:33919)
0807 08:47:55.415908 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:55.415910 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:55.415983 (+ 73us) server_negotiation.cc:407] Connection header received
0807 08:47:55.416033 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:55.416033 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:55.416037 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:55.416073 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:47:55.416519 (+ 446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:47:55.416520 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:47:55.416520 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:47:55.416522 (+ 2us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:47:55.416532 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:47:55.416534 (+ 2us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:47:55.416566 (+ 32us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:55.416596 (+ 30us) server_negotiation.cc:299] Negotiation successful
0807 08:47:55.416606 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 08:47:55.418139 11442 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38234:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 08:47:55.419610 11442 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:47:55.423038 12133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:55.421763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53052 (local address 127.4.159.129:39787)
0807 08:47:55.421940 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:55.421944 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:55.421954 (+ 10us) server_negotiation.cc:407] Connection header received
0807 08:47:55.421981 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:55.421983 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:55.421991 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:55.422265 (+ 274us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:47:55.422763 (+ 498us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:47:55.422768 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:47:55.422768 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:47:55.422771 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:47:55.422799 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:47:55.422813 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:47:55.422866 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:55.422941 (+ 75us) server_negotiation.cc:299] Negotiation successful
0807 08:47:55.422956 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":109,"threads_started":1}
I20260807 08:47:55.423038 12128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:55.421429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.129:39787 (local address 127.0.0.1:53052)
0807 08:47:55.421722 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0807 08:47:55.421732 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 08:47:55.421779 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:47:55.422095 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:47:55.422096 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:47:55.422098 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:47:55.422468 (+ 370us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 08:47:55.422469 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:47:55.422483 (+ 14us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 08:47:55.422503 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:47:55.422867 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:47:55.422875 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:47:55.422878 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 08:47:55.422934 (+ 56us) client_negotiation.cc:240] Negotiation successful
0807 08:47:55.422947 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":93,"threads_started":1}
I20260807 08:47:55.423300 12127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:55.421422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.131:32987 (local address 127.0.0.1:59348)
0807 08:47:55.421659 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0807 08:47:55.421696 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 08:47:55.421753 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:47:55.422359 (+ 606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:47:55.422360 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:47:55.422363 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:47:55.422669 (+ 306us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 08:47:55.422670 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:47:55.422676 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 08:47:55.422689 (+ 13us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:47:55.423066 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:47:55.423067 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:47:55.423068 (+ 1us) client_negotiation.cc:769] Sending connection context
0807 08:47:55.423237 (+ 169us) client_negotiation.cc:240] Negotiation successful
0807 08:47:55.423249 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":47,"threads_started":1}
I20260807 08:47:55.423308 12130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:55.421504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60130 (local address 127.4.159.130:37603)
0807 08:47:55.421664 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:55.421668 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:55.421833 (+ 165us) server_negotiation.cc:407] Connection header received
0807 08:47:55.421862 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:55.421864 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:55.421874 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:55.421937 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:47:55.422777 (+ 840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:47:55.422784 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:47:55.422784 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:47:55.422789 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:47:55.422831 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:47:55.422850 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:47:55.422902 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:55.423205 (+ 303us) server_negotiation.cc:299] Negotiation successful
0807 08:47:55.423224 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":67,"threads_started":1}
I20260807 08:47:55.423050 12129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:55.421421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.130:37603 (local address 127.0.0.1:60130)
0807 08:47:55.421675 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0807 08:47:55.421698 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 08:47:55.421765 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:47:55.421949 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:47:55.421953 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:47:55.421960 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:47:55.422685 (+ 725us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 08:47:55.422685 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:47:55.422691 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 08:47:55.422702 (+ 11us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:47:55.422904 (+ 202us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:47:55.422905 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:47:55.422905 (+ 0us) client_negotiation.cc:769] Sending connection context
0807 08:47:55.422949 (+ 44us) client_negotiation.cc:240] Negotiation successful
0807 08:47:55.422961 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"mutex_wait_us":3,"thread_start_us":88,"threads_started":1}
I20260807 08:47:55.423475 12131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:55.421599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59348 (local address 127.4.159.131:32987)
0807 08:47:55.421983 (+ 384us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:55.421986 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:55.421994 (+ 8us) server_negotiation.cc:407] Connection header received
0807 08:47:55.422014 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:55.422017 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:55.422022 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:55.422087 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:47:55.422934 (+ 847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:47:55.422940 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:47:55.422940 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:47:55.422944 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:47:55.422972 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:47:55.422991 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:47:55.423041 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:55.423391 (+ 350us) server_negotiation.cc:299] Negotiation successful
0807 08:47:55.423407 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":72,"threads_started":1}
I20260807 08:47:55.423746 11630 tablet_service.cc:1467] Processing CreateTablet for tablet ebb9b226e10c496a80bc7e368eac9157 (DEFAULT_TABLE table=test-table [id=abead38727cc41c2bbfefb65c1b9f4c7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:47:55.424007 11630 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ebb9b226e10c496a80bc7e368eac9157. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:47:55.423950 11832 tablet_service.cc:1467] Processing CreateTablet for tablet ebb9b226e10c496a80bc7e368eac9157 (DEFAULT_TABLE table=test-table [id=abead38727cc41c2bbfefb65c1b9f4c7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:47:55.424196 11832 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ebb9b226e10c496a80bc7e368eac9157. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:47:55.426433 12135 tablet_bootstrap.cc:492] T ebb9b226e10c496a80bc7e368eac9157 P 012426fe79f34934829eae4bddb0e4f4: Bootstrap starting.
I20260807 08:47:55.426481 12140 tablet_bootstrap.cc:492] T ebb9b226e10c496a80bc7e368eac9157 P 0fc304540a354d048158cbb41c21cfd0: Bootstrap starting.
I20260807 08:47:55.426728 12022 tablet_service.cc:1467] Processing CreateTablet for tablet ebb9b226e10c496a80bc7e368eac9157 (DEFAULT_TABLE table=test-table [id=abead38727cc41c2bbfefb65c1b9f4c7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:47:55.427028 12022 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ebb9b226e10c496a80bc7e368eac9157. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:47:55.427057 12140 tablet_bootstrap.cc:654] T ebb9b226e10c496a80bc7e368eac9157 P 0fc304540a354d048158cbb41c21cfd0: Neither blocks nor log segments found. Creating new log.
I20260807 08:47:55.427175 12135 tablet_bootstrap.cc:654] T ebb9b226e10c496a80bc7e368eac9157 P 012426fe79f34934829eae4bddb0e4f4: Neither blocks nor log segments found. Creating new log.
I20260807 08:47:55.427276 12140 log.cc:824] T ebb9b226e10c496a80bc7e368eac9157 P 0fc304540a354d048158cbb41c21cfd0: Log is configured to *not* fsync() on all Append() calls
I20260807 08:47:55.427466 12135 log.cc:824] T ebb9b226e10c496a80bc7e368eac9157 P 012426fe79f34934829eae4bddb0e4f4: Log is configured to *not* fsync() on all Append() calls
I20260807 08:47:55.427920 12140 tablet_bootstrap.cc:492] T ebb9b226e10c496a80bc7e368eac9157 P 0fc304540a354d048158cbb41c21cfd0: No bootstrap required, opened a new log
I20260807 08:47:55.427969 12140 ts_tablet_manager.cc:1397] T ebb9b226e10c496a80bc7e368eac9157 P 0fc304540a354d048158cbb41c21cfd0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 08:47:55.428184 12135 tablet_bootstrap.cc:492] T ebb9b226e10c496a80bc7e368eac9157 P 012426fe79f34934829eae4bddb0e4f4: No bootstrap required, opened a new log
I20260807 08:47:55.428243 12135 ts_tablet_manager.cc:1397] T ebb9b226e10c496a80bc7e368eac9157 P 012426fe79f34934829eae4bddb0e4f4: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260807 08:47:55.429754 12140 raft_consensus.cc:348] T ebb9b226e10c496a80bc7e368eac9157 P 0fc304540a354d048158cbb41c21cfd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fc304540a354d048158cbb41c21cfd0" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 37603 } } peers { permanent_uuid: "012426fe79f34934829eae4bddb0e4f4" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 39787 } } peers { permanent_uuid: "a15b209594fc406896b6461c06f4cd28" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 32987 } }
I20260807 08:47:55.429901 12140 raft_consensus.cc:374] T ebb9b226e10c496a80bc7e368eac9157 P 0fc304540a354d048158cbb41c21cfd0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:47:55.429925 12143 tablet_bootstrap.cc:492] T ebb9b226e10c496a80bc7e368eac9157 P a15b209594fc406896b6461c06f4cd28: Bootstrap starting.
I20260807 08:47:55.429940 12140 raft_consensus.cc:729] T ebb9b226e10c496a80bc7e368eac9157 P 0fc304540a354d048158cbb41c21cfd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fc304540a354d048158cbb41c21cfd0, State: Initialized, Role: FOLLOWER
I20260807 08:47:55.430064 12140 consensus_queue.cc:260] T ebb9b226e10c496a80bc7e368eac9157 P 0fc304540a354d048158cbb41c21cfd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fc304540a354d048158cbb41c21cfd0" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 37603 } } peers { permanent_uuid: "012426fe79f34934829eae4bddb0e4f4" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 39787 } } peers { permanent_uuid: "a15b209594fc406896b6461c06f4cd28" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 32987 } }
I20260807 08:47:55.430312 12140 ts_tablet_manager.cc:1428] T ebb9b226e10c496a80bc7e368eac9157 P 0fc304540a354d048158cbb41c21cfd0: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260807 08:47:55.430445 11924 heartbeater.cc:503] Master 127.4.159.190:33919 was elected leader, sending a full tablet report...
I20260807 08:47:55.430540 12143 tablet_bootstrap.cc:654] T ebb9b226e10c496a80bc7e368eac9157 P a15b209594fc406896b6461c06f4cd28: Neither blocks nor log segments found. Creating new log.
I20260807 08:47:55.430816 12143 log.cc:824] T ebb9b226e10c496a80bc7e368eac9157 P a15b209594fc406896b6461c06f4cd28: Log is configured to *not* fsync() on all Append() calls
I20260807 08:47:55.430769 12135 raft_consensus.cc:348] T ebb9b226e10c496a80bc7e368eac9157 P 012426fe79f34934829eae4bddb0e4f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fc304540a354d048158cbb41c21cfd0" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 37603 } } peers { permanent_uuid: "012426fe79f34934829eae4bddb0e4f4" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 39787 } } peers { permanent_uuid: "a15b209594fc406896b6461c06f4cd28" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 32987 } }
I20260807 08:47:55.430919 12135 raft_consensus.cc:374] T ebb9b226e10c496a80bc7e368eac9157 P 012426fe79f34934829eae4bddb0e4f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:47:55.430959 12135 raft_consensus.cc:729] T ebb9b226e10c496a80bc7e368eac9157 P 012426fe79f34934829eae4bddb0e4f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 012426fe79f34934829eae4bddb0e4f4, State: Initialized, Role: FOLLOWER
I20260807 08:47:55.431068 12135 consensus_queue.cc:260] T ebb9b226e10c496a80bc7e368eac9157 P 012426fe79f34934829eae4bddb0e4f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fc304540a354d048158cbb41c21cfd0" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 37603 } } peers { permanent_uuid: "012426fe79f34934829eae4bddb0e4f4" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 39787 } } peers { permanent_uuid: "a15b209594fc406896b6461c06f4cd28" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 32987 } }
I20260807 08:47:55.431319 12135 ts_tablet_manager.cc:1428] T ebb9b226e10c496a80bc7e368eac9157 P 012426fe79f34934829eae4bddb0e4f4: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260807 08:47:55.431355 11729 heartbeater.cc:503] Master 127.4.159.190:33919 was elected leader, sending a full tablet report...
I20260807 08:47:55.431839 12143 tablet_bootstrap.cc:492] T ebb9b226e10c496a80bc7e368eac9157 P a15b209594fc406896b6461c06f4cd28: No bootstrap required, opened a new log
I20260807 08:47:55.431907 12143 ts_tablet_manager.cc:1397] T ebb9b226e10c496a80bc7e368eac9157 P a15b209594fc406896b6461c06f4cd28: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 08:47:55.433785 12143 raft_consensus.cc:348] T ebb9b226e10c496a80bc7e368eac9157 P a15b209594fc406896b6461c06f4cd28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fc304540a354d048158cbb41c21cfd0" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 37603 } } peers { permanent_uuid: "012426fe79f34934829eae4bddb0e4f4" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 39787 } } peers { permanent_uuid: "a15b209594fc406896b6461c06f4cd28" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 32987 } }
I20260807 08:47:55.433899 12143 raft_consensus.cc:374] T ebb9b226e10c496a80bc7e368eac9157 P a15b209594fc406896b6461c06f4cd28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:47:55.433918 12143 raft_consensus.cc:729] T ebb9b226e10c496a80bc7e368eac9157 P a15b209594fc406896b6461c06f4cd28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a15b209594fc406896b6461c06f4cd28, State: Initialized, Role: FOLLOWER
I20260807 08:47:55.433998 12143 consensus_queue.cc:260] T ebb9b226e10c496a80bc7e368eac9157 P a15b209594fc406896b6461c06f4cd28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fc304540a354d048158cbb41c21cfd0" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 37603 } } peers { permanent_uuid: "012426fe79f34934829eae4bddb0e4f4" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 39787 } } peers { permanent_uuid: "a15b209594fc406896b6461c06f4cd28" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 32987 } }
I20260807 08:47:55.434211 12143 ts_tablet_manager.cc:1428] T ebb9b226e10c496a80bc7e368eac9157 P a15b209594fc406896b6461c06f4cd28: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260807 08:47:55.434247 12117 heartbeater.cc:503] Master 127.4.159.190:33919 was elected leader, sending a full tablet report...
W20260807 08:47:55.440003 11731 tablet.cc:2369] T ebb9b226e10c496a80bc7e368eac9157 P 012426fe79f34934829eae4bddb0e4f4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 08:47:55.545418 11926 tablet.cc:2369] T ebb9b226e10c496a80bc7e368eac9157 P 0fc304540a354d048158cbb41c21cfd0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 08:47:55.571765 12146 raft_consensus.cc:482] T ebb9b226e10c496a80bc7e368eac9157 P 012426fe79f34934829eae4bddb0e4f4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:47:55.571877 12146 raft_consensus.cc:504] T ebb9b226e10c496a80bc7e368eac9157 P 012426fe79f34934829eae4bddb0e4f4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fc304540a354d048158cbb41c21cfd0" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 37603 } } peers { permanent_uuid: "012426fe79f34934829eae4bddb0e4f4" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 39787 } } peers { permanent_uuid: "a15b209594fc406896b6461c06f4cd28" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 32987 } }
I20260807 08:47:55.572283 12146 leader_election.cc:288] T ebb9b226e10c496a80bc7e368eac9157 P 012426fe79f34934829eae4bddb0e4f4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0fc304540a354d048158cbb41c21cfd0 (127.4.159.130:37603), a15b209594fc406896b6461c06f4cd28 (127.4.159.131:32987)
I20260807 08:47:55.573506 12130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:55.572450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.129:57931 (local address 127.4.159.130:37603)
0807 08:47:55.572501 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:55.572504 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:55.572514 (+ 10us) server_negotiation.cc:407] Connection header received
0807 08:47:55.572546 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:55.572547 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:55.572553 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:55.572614 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:47:55.573271 (+ 657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:47:55.573273 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:47:55.573274 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:47:55.573278 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:47:55.573297 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:47:55.573302 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:47:55.573357 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:55.573420 (+ 63us) server_negotiation.cc:299] Negotiation successful
0807 08:47:55.573433 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 08:47:55.573506 12131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:55.572530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.129:60849 (local address 127.4.159.131:32987)
0807 08:47:55.572592 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:55.572594 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:55.572718 (+ 124us) server_negotiation.cc:407] Connection header received
0807 08:47:55.572742 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:55.572743 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:55.572747 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:55.572789 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:47:55.573298 (+ 509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:47:55.573299 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:47:55.573300 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:47:55.573304 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:47:55.573317 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:47:55.573321 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:47:55.573370 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:55.573419 (+ 49us) server_negotiation.cc:299] Negotiation successful
0807 08:47:55.573433 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260807 08:47:55.573509 11732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:55.572389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.130:37603 (local address 127.4.159.129:57931)
0807 08:47:55.572441 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0807 08:47:55.572454 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:47:55.572500 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:47:55.572681 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:47:55.572682 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:47:55.572684 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:47:55.573168 (+ 484us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 08:47:55.573168 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:47:55.573177 (+ 9us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 08:47:55.573192 (+ 15us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:47:55.573370 (+ 178us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:47:55.573370 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:47:55.573371 (+ 1us) client_negotiation.cc:769] Sending connection context
0807 08:47:55.573414 (+ 43us) client_negotiation.cc:240] Negotiation successful
0807 08:47:55.573431 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33,"mutex_wait_us":2}
I20260807 08:47:55.573520 12148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:55.572466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.131:32987 (local address 127.4.159.129:60849)
0807 08:47:55.572640 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0807 08:47:55.572649 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 08:47:55.572695 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:47:55.572855 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:47:55.572856 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:47:55.572860 (+ 4us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:47:55.573221 (+ 361us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 08:47:55.573221 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:47:55.573226 (+ 5us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 08:47:55.573235 (+ 9us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:47:55.573371 (+ 136us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:47:55.573372 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:47:55.573372 (+ 0us) client_negotiation.cc:769] Sending connection context
0807 08:47:55.573415 (+ 43us) client_negotiation.cc:240] Negotiation successful
0807 08:47:55.573429 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":89,"threads_started":1}
I20260807 08:47:55.573814 11862 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ebb9b226e10c496a80bc7e368eac9157" candidate_uuid: "012426fe79f34934829eae4bddb0e4f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fc304540a354d048158cbb41c21cfd0" is_pre_election: true
I20260807 08:47:55.573967 11862 raft_consensus.cc:2454] T ebb9b226e10c496a80bc7e368eac9157 P 0fc304540a354d048158cbb41c21cfd0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 012426fe79f34934829eae4bddb0e4f4 in term 0.
I20260807 08:47:55.573973 12047 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ebb9b226e10c496a80bc7e368eac9157" candidate_uuid: "012426fe79f34934829eae4bddb0e4f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a15b209594fc406896b6461c06f4cd28" is_pre_election: true
I20260807 08:47:55.574100 12047 raft_consensus.cc:2454] T ebb9b226e10c496a80bc7e368eac9157 P a15b209594fc406896b6461c06f4cd28 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 012426fe79f34934829eae4bddb0e4f4 in term 0.
I20260807 08:47:55.574205 11554 leader_election.cc:302] T ebb9b226e10c496a80bc7e368eac9157 P 012426fe79f34934829eae4bddb0e4f4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 012426fe79f34934829eae4bddb0e4f4, 0fc304540a354d048158cbb41c21cfd0; no voters:
I20260807 08:47:55.574329 12146 raft_consensus.cc:2781] T ebb9b226e10c496a80bc7e368eac9157 P 012426fe79f34934829eae4bddb0e4f4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:47:55.574412 12146 raft_consensus.cc:482] T ebb9b226e10c496a80bc7e368eac9157 P 012426fe79f34934829eae4bddb0e4f4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:47:55.574440 12146 raft_consensus.cc:3037] T ebb9b226e10c496a80bc7e368eac9157 P 012426fe79f34934829eae4bddb0e4f4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:47:55.575109 12146 raft_consensus.cc:504] T ebb9b226e10c496a80bc7e368eac9157 P 012426fe79f34934829eae4bddb0e4f4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fc304540a354d048158cbb41c21cfd0" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 37603 } } peers { permanent_uuid: "012426fe79f34934829eae4bddb0e4f4" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 39787 } } peers { permanent_uuid: "a15b209594fc406896b6461c06f4cd28" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 32987 } }
I20260807 08:47:55.575229 12146 leader_election.cc:288] T ebb9b226e10c496a80bc7e368eac9157 P 012426fe79f34934829eae4bddb0e4f4 [CANDIDATE]: Term 1 election: Requested vote from peers 0fc304540a354d048158cbb41c21cfd0 (127.4.159.130:37603), a15b209594fc406896b6461c06f4cd28 (127.4.159.131:32987)
I20260807 08:47:55.575359 12047 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ebb9b226e10c496a80bc7e368eac9157" candidate_uuid: "012426fe79f34934829eae4bddb0e4f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a15b209594fc406896b6461c06f4cd28"
I20260807 08:47:55.575372 11862 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ebb9b226e10c496a80bc7e368eac9157" candidate_uuid: "012426fe79f34934829eae4bddb0e4f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fc304540a354d048158cbb41c21cfd0"
I20260807 08:47:55.575429 12047 raft_consensus.cc:3037] T ebb9b226e10c496a80bc7e368eac9157 P a15b209594fc406896b6461c06f4cd28 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:47:55.575439 11862 raft_consensus.cc:3037] T ebb9b226e10c496a80bc7e368eac9157 P 0fc304540a354d048158cbb41c21cfd0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:47:55.576092 12047 raft_consensus.cc:2454] T ebb9b226e10c496a80bc7e368eac9157 P a15b209594fc406896b6461c06f4cd28 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 012426fe79f34934829eae4bddb0e4f4 in term 1.
I20260807 08:47:55.576107 11862 raft_consensus.cc:2454] T ebb9b226e10c496a80bc7e368eac9157 P 0fc304540a354d048158cbb41c21cfd0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 012426fe79f34934829eae4bddb0e4f4 in term 1.
I20260807 08:47:55.576272 11554 leader_election.cc:302] T ebb9b226e10c496a80bc7e368eac9157 P 012426fe79f34934829eae4bddb0e4f4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 012426fe79f34934829eae4bddb0e4f4, 0fc304540a354d048158cbb41c21cfd0; no voters:
I20260807 08:47:55.576355 12146 raft_consensus.cc:2781] T ebb9b226e10c496a80bc7e368eac9157 P 012426fe79f34934829eae4bddb0e4f4 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:47:55.576484 12146 raft_consensus.cc:686] T ebb9b226e10c496a80bc7e368eac9157 P 012426fe79f34934829eae4bddb0e4f4 [term 1 LEADER]: Becoming Leader. State: Replica: 012426fe79f34934829eae4bddb0e4f4, State: Running, Role: LEADER
I20260807 08:47:55.576572 12146 consensus_queue.cc:237] T ebb9b226e10c496a80bc7e368eac9157 P 012426fe79f34934829eae4bddb0e4f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fc304540a354d048158cbb41c21cfd0" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 37603 } } peers { permanent_uuid: "012426fe79f34934829eae4bddb0e4f4" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 39787 } } peers { permanent_uuid: "a15b209594fc406896b6461c06f4cd28" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 32987 } }
I20260807 08:47:55.577302 11443 catalog_manager.cc:5495] T ebb9b226e10c496a80bc7e368eac9157 P 012426fe79f34934829eae4bddb0e4f4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 012426fe79f34934829eae4bddb0e4f4 (127.4.159.129). New cstate: current_term: 1 leader_uuid: "012426fe79f34934829eae4bddb0e4f4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fc304540a354d048158cbb41c21cfd0" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 37603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "012426fe79f34934829eae4bddb0e4f4" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 39787 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a15b209594fc406896b6461c06f4cd28" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 32987 } health_report { overall_health: UNKNOWN } } }
I20260807 08:47:55.602032 12133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:55.601238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53056 (local address 127.4.159.129:39787)
0807 08:47:55.601282 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:55.601284 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:55.601291 (+ 7us) server_negotiation.cc:407] Connection header received
0807 08:47:55.601325 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:55.601326 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:55.601336 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:55.601380 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:47:55.601851 (+ 471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:47:55.601852 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:47:55.601852 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:47:55.601856 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:47:55.601870 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:47:55.601874 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:47:55.601917 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:55.601958 (+ 41us) server_negotiation.cc:299] Negotiation successful
0807 08:47:55.601969 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 08:47:55.603822 11862 raft_consensus.cc:1261] T ebb9b226e10c496a80bc7e368eac9157 P 0fc304540a354d048158cbb41c21cfd0 [term 1 FOLLOWER]: Refusing update from remote peer 012426fe79f34934829eae4bddb0e4f4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:47:55.603822 12047 raft_consensus.cc:1261] T ebb9b226e10c496a80bc7e368eac9157 P a15b209594fc406896b6461c06f4cd28 [term 1 FOLLOWER]: Refusing update from remote peer 012426fe79f34934829eae4bddb0e4f4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:47:55.604184 12149 consensus_queue.cc:1035] T ebb9b226e10c496a80bc7e368eac9157 P 012426fe79f34934829eae4bddb0e4f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a15b209594fc406896b6461c06f4cd28" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 32987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:47:55.604327 12146 consensus_queue.cc:1035] T ebb9b226e10c496a80bc7e368eac9157 P 012426fe79f34934829eae4bddb0e4f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0fc304540a354d048158cbb41c21cfd0" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 37603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:47:55.606773 12155 mvcc.cc:205] Tried to move back new op lower bound from 7315834780070445056 to 7315834779962081280. Current Snapshot: MvccSnapshot[applied={T|T < 7315834780070445056}]
I20260807 08:47:55.608703 11442 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38234:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:47:55.608784 11442 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38234:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:47:55.609436 11442 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 03c2eb4951344907a5f727408b58ac4c: Sending DeleteTablet for 3 replicas of tablet ebb9b226e10c496a80bc7e368eac9157
I20260807 08:47:55.609736 11630 tablet_service.cc:1514] Processing DeleteTablet for tablet ebb9b226e10c496a80bc7e368eac9157 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:47:55 UTC) from {username='slave'} at 127.0.0.1:53052
I20260807 08:47:55.609760 12022 tablet_service.cc:1514] Processing DeleteTablet for tablet ebb9b226e10c496a80bc7e368eac9157 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:47:55 UTC) from {username='slave'} at 127.0.0.1:59348
I20260807 08:47:55.609759 11832 tablet_service.cc:1514] Processing DeleteTablet for tablet ebb9b226e10c496a80bc7e368eac9157 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:47:55 UTC) from {username='slave'} at 127.0.0.1:60130
I20260807 08:47:55.610036 12173 tablet_replica.cc:330] T ebb9b226e10c496a80bc7e368eac9157 P a15b209594fc406896b6461c06f4cd28: stopping tablet replica
I20260807 08:47:55.610131 12173 raft_consensus.cc:2229] T ebb9b226e10c496a80bc7e368eac9157 P a15b209594fc406896b6461c06f4cd28 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:47:55.610189 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 11531
I20260807 08:47:55.610226 12173 raft_consensus.cc:2258] T ebb9b226e10c496a80bc7e368eac9157 P a15b209594fc406896b6461c06f4cd28 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:47:55.610750 12174 tablet_replica.cc:330] T ebb9b226e10c496a80bc7e368eac9157 P 0fc304540a354d048158cbb41c21cfd0: stopping tablet replica
I20260807 08:47:55.610834 12174 raft_consensus.cc:2229] T ebb9b226e10c496a80bc7e368eac9157 P 0fc304540a354d048158cbb41c21cfd0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:47:55.610903 12174 raft_consensus.cc:2258] T ebb9b226e10c496a80bc7e368eac9157 P 0fc304540a354d048158cbb41c21cfd0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:47:55.617894 12173 ts_tablet_manager.cc:1905] T ebb9b226e10c496a80bc7e368eac9157 P a15b209594fc406896b6461c06f4cd28: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:47:55.619732 12173 ts_tablet_manager.cc:1918] T ebb9b226e10c496a80bc7e368eac9157 P a15b209594fc406896b6461c06f4cd28: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:47:55.619810 12173 log.cc:1196] T ebb9b226e10c496a80bc7e368eac9157 P a15b209594fc406896b6461c06f4cd28: Deleting WAL directory at /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786092453486433-4734-0/minicluster-data/ts-2/wal/wals/ebb9b226e10c496a80bc7e368eac9157
I20260807 08:47:55.619838 12174 ts_tablet_manager.cc:1905] T ebb9b226e10c496a80bc7e368eac9157 P 0fc304540a354d048158cbb41c21cfd0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:47:55.620050 12173 ts_tablet_manager.cc:1939] T ebb9b226e10c496a80bc7e368eac9157 P a15b209594fc406896b6461c06f4cd28: Deleting consensus metadata
I20260807 08:47:55.620455 11425 catalog_manager.cc:4841] TS a15b209594fc406896b6461c06f4cd28 (127.4.159.131:32987): tablet ebb9b226e10c496a80bc7e368eac9157 (table test-table [id=abead38727cc41c2bbfefb65c1b9f4c7]) successfully deleted
I20260807 08:47:55.621136 12174 ts_tablet_manager.cc:1918] T ebb9b226e10c496a80bc7e368eac9157 P 0fc304540a354d048158cbb41c21cfd0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:47:55.621201 12174 log.cc:1196] T ebb9b226e10c496a80bc7e368eac9157 P 0fc304540a354d048158cbb41c21cfd0: Deleting WAL directory at /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786092453486433-4734-0/minicluster-data/ts-1/wal/wals/ebb9b226e10c496a80bc7e368eac9157
I20260807 08:47:55.621395 12174 ts_tablet_manager.cc:1939] T ebb9b226e10c496a80bc7e368eac9157 P 0fc304540a354d048158cbb41c21cfd0: Deleting consensus metadata
I20260807 08:47:55.621778 11429 catalog_manager.cc:4841] TS 0fc304540a354d048158cbb41c21cfd0 (127.4.159.130:37603): tablet ebb9b226e10c496a80bc7e368eac9157 (table test-table [id=abead38727cc41c2bbfefb65c1b9f4c7]) successfully deleted
W20260807 08:47:55.623550 11424 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.159.129:39787 (error 108)
I20260807 08:47:55.624012 12129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:55.623863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.129:39787 (local address 127.0.0.1:53062)
0807 08:47:55.623935 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0807 08:47:55.623984 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.129:39787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260807 08:47:55.624079 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 11733
W20260807 08:47:55.624146 11424 catalog_manager.cc:4584] TS 012426fe79f34934829eae4bddb0e4f4 (127.4.159.129:39787): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ebb9b226e10c496a80bc7e368eac9157: Network error: Client connection negotiation failed: client connection to 127.4.159.129:39787: connect: Connection refused (error 111)
I20260807 08:47:55.638481 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 11930
I20260807 08:47:55.647137 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 11398
2026-08-07T08:47:55Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (721 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260807 08:47:55.668810 4734 mini_oidc.cc:152] Starting JWKS server
I20260807 08:47:55.669879 4734 webserver.cc:460] Webserver started at https://127.0.0.1:36749/ using document root <none> and password file <none>
I20260807 08:47:55.669968 4734 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 08:47:55.670145 4734 webserver.cc:460] Webserver started at http://127.0.0.1:36075/ using document root <none> and password file <none>
2026-08-07T08:47:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:47:55Z Disabled control of system clock
I20260807 08:47:55.677742 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786092453486433-4734-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786092453486433-4734-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786092453486433-4734-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.159.190:35357
--webserver_interface=127.4.159.190
--webserver_port=0
--builtin_ntp_servers=127.4.159.148:38031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.159.190:35357
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36749/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786092453486433-4734-0/testchainca.cert with env {}
W20260807 08:47:55.757437 12185 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:47:55.757623 12185 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:47:55.757647 12185 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:47:55.758755 12185 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:47:55.758790 12185 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:47:55.758801 12185 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:47:55.758812 12185 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:47:55.758822 12185 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 08:47:55.758834 12185 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 08:47:55.758843 12185 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:36749/jwks.json
I20260807 08:47:55.760330 12185 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:38031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786092453486433-4734-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786092453486433-4734-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.159.190:35357
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.159.190:35357
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36749/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.159.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786092453486433-4734-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12185
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786092453486433-4734-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:47:55.760526 12185 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:47:55.760687 12185 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:47:55.765146 12191 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:47:55.765161 12195 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:47:55.765223 12193 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:47:55.765357 12185 server_base.cc:1034] running on GCE node
I20260807 08:47:55.765658 12185 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:47:55.765910 12185 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:47:55.767042 12185 hybrid_clock.cc:648] HybridClock initialized: now 1786092475767027 us; error 29 us; skew 500 ppm
I20260807 08:47:55.768332 12185 webserver.cc:460] Webserver started at http://127.4.159.190:33129/ using document root <none> and password file <none>
I20260807 08:47:55.768563 12185 fs_manager.cc:359] Metadata directory not provided
I20260807 08:47:55.768625 12185 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:47:55.768752 12185 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:47:55.769776 12185 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786092453486433-4734-0/minicluster-data/master-0/data/instance:
uuid: "1bab42e0f0d04536b400b764538b9bd5"
format_stamp: "Formatted at 2026-08-07 08:47:55 on dist-test-slave-6zbq"
server_key: "5517fe3c9b328b5f7894bf5d62340a69"
server_key_iv: "cd3e0ea980a01ce1f44d996145a84638"
server_key_version: "encryptionkey@0"
I20260807 08:47:55.770107 12185 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786092453486433-4734-0/minicluster-data/master-0/wal/instance:
uuid: "1bab42e0f0d04536b400b764538b9bd5"
format_stamp: "Formatted at 2026-08-07 08:47:55 on dist-test-slave-6zbq"
server_key: "5517fe3c9b328b5f7894bf5d62340a69"
server_key_iv: "cd3e0ea980a01ce1f44d996145a84638"
server_key_version: "encryptionkey@0"
I20260807 08:47:55.771404 12185 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 08:47:55.772054 12201 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:55.772189 12185 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 08:47:55.772272 12185 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786092453486433-4734-0/minicluster-data/master-0/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786092453486433-4734-0/minicluster-data/master-0/wal
uuid: "1bab42e0f0d04536b400b764538b9bd5"
format_stamp: "Formatted at 2026-08-07 08:47:55 on dist-test-slave-6zbq"
server_key: "5517fe3c9b328b5f7894bf5d62340a69"
server_key_iv: "cd3e0ea980a01ce1f44d996145a84638"
server_key_version: "encryptionkey@0"
I20260807 08:47:55.772332 12185 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786092453486433-4734-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786092453486433-4734-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786092453486433-4734-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:47:55.787402 12185 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:47:55.787681 12185 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:47:55.787801 12185 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:47:55.791667 12185 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.190:35357
I20260807 08:47:55.791699 12283 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.190:35357 every 8 connection(s)
I20260807 08:47:55.791990 12185 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
I20260807 08:47:55.792925 12284 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:47:55.795096 12284 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1bab42e0f0d04536b400b764538b9bd5: Bootstrap starting.
I20260807 08:47:55.795655 12284 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1bab42e0f0d04536b400b764538b9bd5: Neither blocks nor log segments found. Creating new log.
I20260807 08:47:55.795879 12284 log.cc:824] T 00000000000000000000000000000000 P 1bab42e0f0d04536b400b764538b9bd5: Log is configured to *not* fsync() on all Append() calls
I20260807 08:47:55.796531 12284 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1bab42e0f0d04536b400b764538b9bd5: No bootstrap required, opened a new log
I20260807 08:47:55.798027 12284 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1bab42e0f0d04536b400b764538b9bd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1bab42e0f0d04536b400b764538b9bd5" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 35357 } }
I20260807 08:47:55.798159 12284 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1bab42e0f0d04536b400b764538b9bd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:47:55.798182 12284 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1bab42e0f0d04536b400b764538b9bd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bab42e0f0d04536b400b764538b9bd5, State: Initialized, Role: FOLLOWER
I20260807 08:47:55.798290 12284 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1bab42e0f0d04536b400b764538b9bd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1bab42e0f0d04536b400b764538b9bd5" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 35357 } }
I20260807 08:47:55.798369 12284 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1bab42e0f0d04536b400b764538b9bd5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:47:55.798409 12284 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1bab42e0f0d04536b400b764538b9bd5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:47:55.798447 12284 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1bab42e0f0d04536b400b764538b9bd5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:47:55.798903 12284 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1bab42e0f0d04536b400b764538b9bd5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1bab42e0f0d04536b400b764538b9bd5" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 35357 } }
I20260807 08:47:55.798987 12284 leader_election.cc:302] T 00000000000000000000000000000000 P 1bab42e0f0d04536b400b764538b9bd5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1bab42e0f0d04536b400b764538b9bd5; no voters:
I20260807 08:47:55.799110 12284 leader_election.cc:288] T 00000000000000000000000000000000 P 1bab42e0f0d04536b400b764538b9bd5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:47:55.799167 12291 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1bab42e0f0d04536b400b764538b9bd5 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:47:55.799314 12291 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1bab42e0f0d04536b400b764538b9bd5 [term 1 LEADER]: Becoming Leader. State: Replica: 1bab42e0f0d04536b400b764538b9bd5, State: Running, Role: LEADER
I20260807 08:47:55.799351 12284 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1bab42e0f0d04536b400b764538b9bd5 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:47:55.799412 12291 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1bab42e0f0d04536b400b764538b9bd5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1bab42e0f0d04536b400b764538b9bd5" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 35357 } }
I20260807 08:47:55.799751 12293 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1bab42e0f0d04536b400b764538b9bd5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1bab42e0f0d04536b400b764538b9bd5. Latest consensus state: current_term: 1 leader_uuid: "1bab42e0f0d04536b400b764538b9bd5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1bab42e0f0d04536b400b764538b9bd5" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 35357 } } }
I20260807 08:47:55.799830 12292 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1bab42e0f0d04536b400b764538b9bd5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1bab42e0f0d04536b400b764538b9bd5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1bab42e0f0d04536b400b764538b9bd5" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 35357 } } }
I20260807 08:47:55.799840 12293 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1bab42e0f0d04536b400b764538b9bd5 [sys.catalog]: This master's current role is: LEADER
I20260807 08:47:55.799906 12292 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1bab42e0f0d04536b400b764538b9bd5 [sys.catalog]: This master's current role is: LEADER
I20260807 08:47:55.800185 12303 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:47:55.800693 12303 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:47:55.801477 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 12185
I20260807 08:47:55.801561 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786092453486433-4734-0/minicluster-data/master-0/wal/instance
I20260807 08:47:55.801746 4734 external_mini_cluster.cc:1428] Setting key 7f3dd416b118a17552be9577481e2043
I20260807 08:47:55.802245 12303 catalog_manager.cc:1347] Generated new cluster ID: 216c442b26fd48bd94623ad3cff2045e
I20260807 08:47:55.802299 12303 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:47:55.817524 12303 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:47:55.817997 12303 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:47:55.820003 12318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:55.802153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48196 (local address 127.4.159.190:35357)
0807 08:47:55.802358 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:55.802367 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:55.814555 (+ 12188us) server_negotiation.cc:407] Connection header received
0807 08:47:55.814696 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:55.814703 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:55.814774 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:55.814901 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:47:55.815149 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:55.816518 (+ 1369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:55.817650 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:55.818297 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:55.818474 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:55.819159 (+ 685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:47:55.819164 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:47:55.819164 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:47:55.819376 (+ 212us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:47:55.819420 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:47:55.819435 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:47:55.819505 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:55.819578 (+ 73us) server_negotiation.cc:299] Negotiation successful
0807 08:47:55.819682 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":91,"threads_started":1}
I20260807 08:47:55.821271 4734 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 08:47:55.821475 12303 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1bab42e0f0d04536b400b764538b9bd5: Generated new TSK 0
I20260807 08:47:55.821640 12303 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260807 08:47:55.842515 12327 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260807 08:47:55.842599 12327 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260807 08:47:55.842890 12318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:55.827611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48206 (local address 127.4.159.190:35357)
0807 08:47:55.827657 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:55.827659 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:55.827740 (+ 81us) server_negotiation.cc:407] Connection header received
0807 08:47:55.827805 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:55.827806 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:55.827817 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:55.827886 (+ 69us) server_negotiation.cc:226] Negotiated authn=JWT
0807 08:47:55.828003 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:55.828760 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:55.829476 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:55.829771 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:55.829835 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:55.829980 (+ 145us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 08:47:55.842667 (+ 12687us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36749/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0807 08:47:55.842821 (+ 154us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:48206: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36749/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":24}
W20260807 08:47:55.842947 12318 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:48206: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36749/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260807 08:47:55.843237 12326 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:47:55.827513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:35357 (local address 127.0.0.1:48206)
0807 08:47:55.827675 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0807 08:47:55.827685 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 08:47:55.827741 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:47:55.827887 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:47:55.827888 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:47:55.827889 (+ 1us) client_negotiation.cc:189] Negotiated authn=JWT
0807 08:47:55.827950 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:55.827952 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:55.828836 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:55.828837 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:55.829408 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:55.829409 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:55.829837 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:55.829837 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:55.829876 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:55.829897 (+ 21us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 08:47:55.842781 (+ 12884us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36749/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0807 08:47:55.842892 (+ 111us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.4.159.190:35357: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36749/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":74,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.4.159.190:35357: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36749/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260807 08:47:55.843660 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 12185
2026-08-07T08:47:55Z chronyd exiting
I20260807 08:47:55.888175 4734 test_util.cc:175] -----------------------------------------------
I20260807 08:47:55.888250 4734 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786092453486433-4734-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (220 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260807 08:47:55.888962 4734 mini_oidc.cc:152] Starting JWKS server
I20260807 08:47:55.889609 4734 webserver.cc:460] Webserver started at https://127.0.0.1:39377/ using document root <none> and password file <none>
I20260807 08:47:55.889653 4734 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 08:47:55.889783 4734 webserver.cc:460] Webserver started at http://127.0.0.1:46309/ using document root <none> and password file <none>
2026-08-07T08:47:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:47:55Z Disabled control of system clock
I20260807 08:47:55.897861 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786092453486433-4734-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786092453486433-4734-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786092453486433-4734-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.159.190:43449
--webserver_interface=127.4.159.190
--webserver_port=0
--builtin_ntp_servers=127.4.159.148:42359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.159.190:43449
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39377/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786092453486433-4734-0/testchainca.cert with env {}
W20260807 08:47:55.977494 12333 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:47:55.977698 12333 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:47:55.977727 12333 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:47:55.978884 12333 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:47:55.978925 12333 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:47:55.978942 12333 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:47:55.978957 12333 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:47:55.978972 12333 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 08:47:55.978988 12333 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 08:47:55.979008 12333 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:39377/jwks.json
I20260807 08:47:55.980508 12333 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:42359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786092453486433-4734-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786092453486433-4734-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.159.190:43449
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.159.190:43449
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39377/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.159.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786092453486433-4734-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12333
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786092453486433-4734-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:47:55.980751 12333 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:47:55.980937 12333 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:47:55.985356 12341 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:47:55.985405 12344 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:47:55.985400 12347 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:47:55.985440 12333 server_base.cc:1034] running on GCE node
I20260807 08:47:55.985744 12333 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:47:55.986023 12333 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:47:55.987164 12333 hybrid_clock.cc:648] HybridClock initialized: now 1786092475987162 us; error 44 us; skew 500 ppm
I20260807 08:47:55.988284 12333 webserver.cc:460] Webserver started at http://127.4.159.190:46561/ using document root <none> and password file <none>
I20260807 08:47:55.988485 12333 fs_manager.cc:359] Metadata directory not provided
I20260807 08:47:55.988533 12333 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:47:55.988651 12333 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:47:55.989648 12333 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786092453486433-4734-0/minicluster-data/master-0/data/instance:
uuid: "f330522046cd4fc4a3bba61d67ae5216"
format_stamp: "Formatted at 2026-08-07 08:47:55 on dist-test-slave-6zbq"
server_key: "287906255cd4c4a0694fadab24e74526"
server_key_iv: "d2b7e4dde00d0162a88a9575a41d2177"
server_key_version: "encryptionkey@0"
I20260807 08:47:55.989974 12333 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786092453486433-4734-0/minicluster-data/master-0/wal/instance:
uuid: "f330522046cd4fc4a3bba61d67ae5216"
format_stamp: "Formatted at 2026-08-07 08:47:55 on dist-test-slave-6zbq"
server_key: "287906255cd4c4a0694fadab24e74526"
server_key_iv: "d2b7e4dde00d0162a88a9575a41d2177"
server_key_version: "encryptionkey@0"
I20260807 08:47:55.991190 12333 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 08:47:55.991849 12360 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:55.991988 12333 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 08:47:55.992069 12333 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786092453486433-4734-0/minicluster-data/master-0/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786092453486433-4734-0/minicluster-data/master-0/wal
uuid: "f330522046cd4fc4a3bba61d67ae5216"
format_stamp: "Formatted at 2026-08-07 08:47:55 on dist-test-slave-6zbq"
server_key: "287906255cd4c4a0694fadab24e74526"
server_key_iv: "d2b7e4dde00d0162a88a9575a41d2177"
server_key_version: "encryptionkey@0"
I20260807 08:47:55.992128 12333 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786092453486433-4734-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786092453486433-4734-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786092453486433-4734-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:47:55.998173 12333 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:47:55.998446 12333 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:47:55.998564 12333 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:47:56.002589 12333 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.190:43449
I20260807 08:47:56.002635 12442 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.190:43449 every 8 connection(s)
I20260807 08:47:56.002918 12333 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
I20260807 08:47:56.003814 12444 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:47:56.005841 12444 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f330522046cd4fc4a3bba61d67ae5216: Bootstrap starting.
I20260807 08:47:56.006381 12444 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f330522046cd4fc4a3bba61d67ae5216: Neither blocks nor log segments found. Creating new log.
I20260807 08:47:56.006635 12444 log.cc:824] T 00000000000000000000000000000000 P f330522046cd4fc4a3bba61d67ae5216: Log is configured to *not* fsync() on all Append() calls
I20260807 08:47:56.007225 12444 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f330522046cd4fc4a3bba61d67ae5216: No bootstrap required, opened a new log
I20260807 08:47:56.008710 12444 raft_consensus.cc:348] T 00000000000000000000000000000000 P f330522046cd4fc4a3bba61d67ae5216 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f330522046cd4fc4a3bba61d67ae5216" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 43449 } }
I20260807 08:47:56.008836 12444 raft_consensus.cc:374] T 00000000000000000000000000000000 P f330522046cd4fc4a3bba61d67ae5216 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:47:56.008855 12444 raft_consensus.cc:729] T 00000000000000000000000000000000 P f330522046cd4fc4a3bba61d67ae5216 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f330522046cd4fc4a3bba61d67ae5216, State: Initialized, Role: FOLLOWER
I20260807 08:47:56.008956 12444 consensus_queue.cc:260] T 00000000000000000000000000000000 P f330522046cd4fc4a3bba61d67ae5216 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f330522046cd4fc4a3bba61d67ae5216" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 43449 } }
I20260807 08:47:56.009030 12444 raft_consensus.cc:388] T 00000000000000000000000000000000 P f330522046cd4fc4a3bba61d67ae5216 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:47:56.009063 12444 raft_consensus.cc:482] T 00000000000000000000000000000000 P f330522046cd4fc4a3bba61d67ae5216 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:47:56.009094 12444 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f330522046cd4fc4a3bba61d67ae5216 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:47:56.009534 12444 raft_consensus.cc:504] T 00000000000000000000000000000000 P f330522046cd4fc4a3bba61d67ae5216 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f330522046cd4fc4a3bba61d67ae5216" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 43449 } }
I20260807 08:47:56.009616 12444 leader_election.cc:302] T 00000000000000000000000000000000 P f330522046cd4fc4a3bba61d67ae5216 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f330522046cd4fc4a3bba61d67ae5216; no voters:
I20260807 08:47:56.009740 12444 leader_election.cc:288] T 00000000000000000000000000000000 P f330522046cd4fc4a3bba61d67ae5216 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:47:56.009814 12447 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f330522046cd4fc4a3bba61d67ae5216 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:47:56.009943 12444 sys_catalog.cc:564] T 00000000000000000000000000000000 P f330522046cd4fc4a3bba61d67ae5216 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:47:56.009975 12447 raft_consensus.cc:686] T 00000000000000000000000000000000 P f330522046cd4fc4a3bba61d67ae5216 [term 1 LEADER]: Becoming Leader. State: Replica: f330522046cd4fc4a3bba61d67ae5216, State: Running, Role: LEADER
I20260807 08:47:56.010073 12447 consensus_queue.cc:237] T 00000000000000000000000000000000 P f330522046cd4fc4a3bba61d67ae5216 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f330522046cd4fc4a3bba61d67ae5216" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 43449 } }
I20260807 08:47:56.010453 12451 sys_catalog.cc:455] T 00000000000000000000000000000000 P f330522046cd4fc4a3bba61d67ae5216 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f330522046cd4fc4a3bba61d67ae5216. Latest consensus state: current_term: 1 leader_uuid: "f330522046cd4fc4a3bba61d67ae5216" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f330522046cd4fc4a3bba61d67ae5216" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 43449 } } }
I20260807 08:47:56.010548 12449 sys_catalog.cc:455] T 00000000000000000000000000000000 P f330522046cd4fc4a3bba61d67ae5216 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f330522046cd4fc4a3bba61d67ae5216" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f330522046cd4fc4a3bba61d67ae5216" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 43449 } } }
I20260807 08:47:56.010599 12451 sys_catalog.cc:458] T 00000000000000000000000000000000 P f330522046cd4fc4a3bba61d67ae5216 [sys.catalog]: This master's current role is: LEADER
I20260807 08:47:56.010638 12449 sys_catalog.cc:458] T 00000000000000000000000000000000 P f330522046cd4fc4a3bba61d67ae5216 [sys.catalog]: This master's current role is: LEADER
I20260807 08:47:56.010813 12459 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:47:56.011328 12459 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:47:56.011715 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 12333
I20260807 08:47:56.011801 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786092453486433-4734-0/minicluster-data/master-0/wal/instance
I20260807 08:47:56.011970 4734 external_mini_cluster.cc:1428] Setting key 02532c0f76feee8a436587810ecd6f0c
I20260807 08:47:56.013450 12459 catalog_manager.cc:1347] Generated new cluster ID: 9263100924214480a70f65a5f598b82f
I20260807 08:47:56.013502 12459 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:47:56.018584 12475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:56.012492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37370 (local address 127.4.159.190:43449)
0807 08:47:56.012910 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:56.012919 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:56.012950 (+ 31us) server_negotiation.cc:407] Connection header received
0807 08:47:56.013043 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:56.013048 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:56.013115 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:56.013263 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:47:56.013510 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:56.014955 (+ 1445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:56.016078 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:56.016701 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:56.016813 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:56.017642 (+ 829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:47:56.017646 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:47:56.017646 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:47:56.017846 (+ 200us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:47:56.017889 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:47:56.017905 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:47:56.017964 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:56.018157 (+ 193us) server_negotiation.cc:299] Negotiation successful
0807 08:47:56.018246 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349,"thread_start_us":60,"threads_started":1}
I20260807 08:47:56.019546 4734 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 08:47:56.032030 12459 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:47:56.032475 12459 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:47:56.035040 12459 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f330522046cd4fc4a3bba61d67ae5216: Generated new TSK 0
I20260807 08:47:56.035187 12459 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260807 08:47:56.055855 12489 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260807 08:47:56.055931 12489 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260807 08:47:56.056208 12475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:56.042313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37382 (local address 127.4.159.190:43449)
0807 08:47:56.042369 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:56.042371 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:56.042416 (+ 45us) server_negotiation.cc:407] Connection header received
0807 08:47:56.042499 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:56.042500 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:56.042519 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:56.042571 (+ 52us) server_negotiation.cc:226] Negotiated authn=JWT
0807 08:47:56.042693 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:56.043151 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:56.044233 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:56.044457 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:56.044518 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:56.044667 (+ 149us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 08:47:56.056033 (+ 11366us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39377/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0807 08:47:56.056153 (+ 120us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:37382: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39377/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":19}
W20260807 08:47:56.056250 12475 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:37382: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39377/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260807 08:47:56.056371 12488 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:47:56.042216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:43449 (local address 127.0.0.1:37382)
0807 08:47:56.042358 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0807 08:47:56.042369 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 08:47:56.042435 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:47:56.042584 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:47:56.042585 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:47:56.042587 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0807 08:47:56.042637 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:56.042639 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:56.043219 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:56.043220 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:56.044138 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:56.044141 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:56.044529 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:56.044530 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:56.044586 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:56.044592 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 08:47:56.056134 (+ 11542us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39377/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0807 08:47:56.056262 (+ 128us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.4.159.190:43449: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39377/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":70,"threads_started":1}
I20260807 08:47:56.056630 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 12333
2026-08-07T08:47:56Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (214 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260807 08:47:56.103034 4734 mini_oidc.cc:152] Starting JWKS server
I20260807 08:47:56.103698 4734 webserver.cc:460] Webserver started at https://127.0.0.1:34225/ using document root <none> and password file <none>
I20260807 08:47:56.103741 4734 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 08:47:56.103883 4734 webserver.cc:460] Webserver started at http://127.0.0.1:37771/ using document root <none> and password file <none>
2026-08-07T08:47:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:47:56Z Disabled control of system clock
I20260807 08:47:56.112113 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786092453486433-4734-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786092453486433-4734-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786092453486433-4734-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.159.190:44451
--webserver_interface=127.4.159.190
--webserver_port=0
--builtin_ntp_servers=127.4.159.148:39143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.159.190:44451
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34225/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260807 08:47:56.195901 12496 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:47:56.196094 12496 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:47:56.196125 12496 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:47:56.197268 12496 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:47:56.197309 12496 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:47:56.197325 12496 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:47:56.197341 12496 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:47:56.197356 12496 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 08:47:56.197373 12496 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 08:47:56.197396 12496 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:34225/jwks.json
I20260807 08:47:56.198807 12496 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:39143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786092453486433-4734-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786092453486433-4734-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.159.190:44451
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.159.190:44451
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34225/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.159.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12496
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786092453486433-4734-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:47:56.199038 12496 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:47:56.199322 12496 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:47:56.204267 12504 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:47:56.204380 12509 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:47:56.204421 12507 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:47:56.204772 12496 server_base.cc:1034] running on GCE node
I20260807 08:47:56.205008 12496 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:47:56.205284 12496 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:47:56.206416 12496 hybrid_clock.cc:648] HybridClock initialized: now 1786092476206400 us; error 31 us; skew 500 ppm
I20260807 08:47:56.207773 12496 webserver.cc:460] Webserver started at http://127.4.159.190:41385/ using document root <none> and password file <none>
I20260807 08:47:56.208057 12496 fs_manager.cc:359] Metadata directory not provided
I20260807 08:47:56.208109 12496 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:47:56.208240 12496 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:47:56.209283 12496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786092453486433-4734-0/minicluster-data/master-0/data/instance:
uuid: "c94971fd3c9047f0b81268dd60660697"
format_stamp: "Formatted at 2026-08-07 08:47:56 on dist-test-slave-6zbq"
server_key: "6e6e2504089279ceb0c84c0e90b1dfca"
server_key_iv: "e1c4682983352a894b439185f86a1264"
server_key_version: "encryptionkey@0"
I20260807 08:47:56.209617 12496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786092453486433-4734-0/minicluster-data/master-0/wal/instance:
uuid: "c94971fd3c9047f0b81268dd60660697"
format_stamp: "Formatted at 2026-08-07 08:47:56 on dist-test-slave-6zbq"
server_key: "6e6e2504089279ceb0c84c0e90b1dfca"
server_key_iv: "e1c4682983352a894b439185f86a1264"
server_key_version: "encryptionkey@0"
I20260807 08:47:56.210877 12496 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 08:47:56.211519 12515 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:56.211676 12496 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 08:47:56.211759 12496 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786092453486433-4734-0/minicluster-data/master-0/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786092453486433-4734-0/minicluster-data/master-0/wal
uuid: "c94971fd3c9047f0b81268dd60660697"
format_stamp: "Formatted at 2026-08-07 08:47:56 on dist-test-slave-6zbq"
server_key: "6e6e2504089279ceb0c84c0e90b1dfca"
server_key_iv: "e1c4682983352a894b439185f86a1264"
server_key_version: "encryptionkey@0"
I20260807 08:47:56.211826 12496 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786092453486433-4734-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786092453486433-4734-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786092453486433-4734-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:47:56.221715 12496 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:47:56.221978 12496 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:47:56.222098 12496 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:47:56.225834 12496 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.190:44451
I20260807 08:47:56.225867 12595 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.190:44451 every 8 connection(s)
I20260807 08:47:56.226159 12496 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
I20260807 08:47:56.226830 12596 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:47:56.229408 12596 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c94971fd3c9047f0b81268dd60660697: Bootstrap starting.
I20260807 08:47:56.229951 12596 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c94971fd3c9047f0b81268dd60660697: Neither blocks nor log segments found. Creating new log.
I20260807 08:47:56.230198 12596 log.cc:824] T 00000000000000000000000000000000 P c94971fd3c9047f0b81268dd60660697: Log is configured to *not* fsync() on all Append() calls
I20260807 08:47:56.230854 12596 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c94971fd3c9047f0b81268dd60660697: No bootstrap required, opened a new log
I20260807 08:47:56.232385 12596 raft_consensus.cc:348] T 00000000000000000000000000000000 P c94971fd3c9047f0b81268dd60660697 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c94971fd3c9047f0b81268dd60660697" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 44451 } }
I20260807 08:47:56.232515 12596 raft_consensus.cc:374] T 00000000000000000000000000000000 P c94971fd3c9047f0b81268dd60660697 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:47:56.232537 12596 raft_consensus.cc:729] T 00000000000000000000000000000000 P c94971fd3c9047f0b81268dd60660697 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c94971fd3c9047f0b81268dd60660697, State: Initialized, Role: FOLLOWER
I20260807 08:47:56.232668 12596 consensus_queue.cc:260] T 00000000000000000000000000000000 P c94971fd3c9047f0b81268dd60660697 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c94971fd3c9047f0b81268dd60660697" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 44451 } }
I20260807 08:47:56.232752 12596 raft_consensus.cc:388] T 00000000000000000000000000000000 P c94971fd3c9047f0b81268dd60660697 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:47:56.232788 12596 raft_consensus.cc:482] T 00000000000000000000000000000000 P c94971fd3c9047f0b81268dd60660697 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:47:56.232836 12596 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c94971fd3c9047f0b81268dd60660697 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:47:56.233315 12596 raft_consensus.cc:504] T 00000000000000000000000000000000 P c94971fd3c9047f0b81268dd60660697 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c94971fd3c9047f0b81268dd60660697" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 44451 } }
I20260807 08:47:56.233402 12596 leader_election.cc:302] T 00000000000000000000000000000000 P c94971fd3c9047f0b81268dd60660697 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c94971fd3c9047f0b81268dd60660697; no voters:
I20260807 08:47:56.233551 12596 leader_election.cc:288] T 00000000000000000000000000000000 P c94971fd3c9047f0b81268dd60660697 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:47:56.233593 12602 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c94971fd3c9047f0b81268dd60660697 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:47:56.233734 12602 raft_consensus.cc:686] T 00000000000000000000000000000000 P c94971fd3c9047f0b81268dd60660697 [term 1 LEADER]: Becoming Leader. State: Replica: c94971fd3c9047f0b81268dd60660697, State: Running, Role: LEADER
I20260807 08:47:56.233793 12596 sys_catalog.cc:564] T 00000000000000000000000000000000 P c94971fd3c9047f0b81268dd60660697 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:47:56.233834 12602 consensus_queue.cc:237] T 00000000000000000000000000000000 P c94971fd3c9047f0b81268dd60660697 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c94971fd3c9047f0b81268dd60660697" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 44451 } }
I20260807 08:47:56.234192 12603 sys_catalog.cc:455] T 00000000000000000000000000000000 P c94971fd3c9047f0b81268dd60660697 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c94971fd3c9047f0b81268dd60660697" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c94971fd3c9047f0b81268dd60660697" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 44451 } } }
I20260807 08:47:56.234278 12603 sys_catalog.cc:458] T 00000000000000000000000000000000 P c94971fd3c9047f0b81268dd60660697 [sys.catalog]: This master's current role is: LEADER
I20260807 08:47:56.234402 12604 sys_catalog.cc:455] T 00000000000000000000000000000000 P c94971fd3c9047f0b81268dd60660697 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c94971fd3c9047f0b81268dd60660697. Latest consensus state: current_term: 1 leader_uuid: "c94971fd3c9047f0b81268dd60660697" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c94971fd3c9047f0b81268dd60660697" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 44451 } } }
I20260807 08:47:56.234467 12604 sys_catalog.cc:458] T 00000000000000000000000000000000 P c94971fd3c9047f0b81268dd60660697 [sys.catalog]: This master's current role is: LEADER
I20260807 08:47:56.234565 12613 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:47:56.235047 12613 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:47:56.236225 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 12496
I20260807 08:47:56.236330 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786092453486433-4734-0/minicluster-data/master-0/wal/instance
I20260807 08:47:56.236502 12613 catalog_manager.cc:1347] Generated new cluster ID: fe03a5255cd345a68674a6db59a810a0
I20260807 08:47:56.236524 4734 external_mini_cluster.cc:1428] Setting key 44440f2e22b853e49ae26624ba9bf5e0
I20260807 08:47:56.236552 12613 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:47:56.242494 12626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:56.237097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36346 (local address 127.4.159.190:44451)
0807 08:47:56.237259 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:56.237267 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:56.237297 (+ 30us) server_negotiation.cc:407] Connection header received
0807 08:47:56.237416 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:56.237423 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:56.237488 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:56.237620 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:47:56.237829 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:56.239259 (+ 1430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:56.240248 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:56.240851 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:56.240969 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:56.241555 (+ 586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:47:56.241559 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:47:56.241559 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:47:56.241768 (+ 209us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:47:56.241810 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:47:56.241824 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:47:56.241892 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:56.241945 (+ 53us) server_negotiation.cc:299] Negotiation successful
0807 08:47:56.242020 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":94,"thread_start_us":69,"threads_started":1}
I20260807 08:47:56.243341 4734 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 08:47:56.252537 12613 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:47:56.252976 12613 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:47:56.257694 12613 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c94971fd3c9047f0b81268dd60660697: Generated new TSK 0
I20260807 08:47:56.257851 12613 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:47:56.266708 12626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:56.263800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36358 (local address 127.4.159.190:44451)
0807 08:47:56.263844 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:56.263846 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:56.263938 (+ 92us) server_negotiation.cc:407] Connection header received
0807 08:47:56.263996 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:56.263997 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:56.264024 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:56.264077 (+ 53us) server_negotiation.cc:226] Negotiated authn=JWT
0807 08:47:56.264198 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:56.264659 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:56.265366 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:56.265564 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:56.265617 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:56.265776 (+ 159us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 08:47:56.266520 (+ 744us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34225/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0807 08:47:56.266640 (+ 120us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:36358: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34225/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":25}
W20260807 08:47:56.266764 12626 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:36358: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34225/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260807 08:47:56.266979 12636 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:47:56.263733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:44451 (local address 127.0.0.1:36358)
0807 08:47:56.263881 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0807 08:47:56.263889 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0807 08:47:56.263943 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:47:56.264082 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:47:56.264083 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:47:56.264085 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0807 08:47:56.264141 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:56.264143 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:56.264730 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:56.264731 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:56.265290 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:56.265292 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:56.265626 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:56.265627 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:56.265693 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:56.265699 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 08:47:56.266614 (+ 915us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34225/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0807 08:47:56.266709 (+ 95us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.4.159.190:44451: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34225/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":76,"threads_started":1}
I20260807 08:47:56.267171 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 12496
E20260807 08:47:56.288887 12638 webserver.cc:533] Webserver: sslize failed:
E20260807 08:47:56.288957 12638 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-07T08:47:56Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (213 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-07T08:47:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:47:56Z Disabled control of system clock
I20260807 08:47:56.324836 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786092453486433-4734-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786092453486433-4734-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786092453486433-4734-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.159.190:46767
--webserver_interface=127.4.159.190
--webserver_port=0
--builtin_ntp_servers=127.4.159.148:35043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.159.190:46767
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786092453486433-4734-0/testchainca.cert with env {}
W20260807 08:47:56.405006 12642 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:47:56.405208 12642 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:47:56.405239 12642 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:47:56.406388 12642 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:47:56.406427 12642 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:47:56.406445 12642 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:47:56.406461 12642 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:47:56.406476 12642 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 08:47:56.406493 12642 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 08:47:56.406514 12642 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260807 08:47:56.407953 12642 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:35043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786092453486433-4734-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786092453486433-4734-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.159.190:46767
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.159.190:46767
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.159.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786092453486433-4734-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12642
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786092453486433-4734-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:47:56.408195 12642 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:47:56.408388 12642 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:47:56.412811 12650 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:47:56.412832 12652 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:47:56.412802 12648 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:47:56.412923 12642 server_base.cc:1034] running on GCE node
I20260807 08:47:56.413200 12642 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:47:56.413477 12642 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:47:56.414635 12642 hybrid_clock.cc:648] HybridClock initialized: now 1786092476414618 us; error 33 us; skew 500 ppm
I20260807 08:47:56.415771 12642 webserver.cc:460] Webserver started at http://127.4.159.190:44077/ using document root <none> and password file <none>
I20260807 08:47:56.415973 12642 fs_manager.cc:359] Metadata directory not provided
I20260807 08:47:56.416043 12642 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:47:56.416172 12642 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:47:56.417251 12642 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786092453486433-4734-0/minicluster-data/master-0/data/instance:
uuid: "93ba8fc84a3347eb8bd89f6f85efcedd"
format_stamp: "Formatted at 2026-08-07 08:47:56 on dist-test-slave-6zbq"
server_key: "4102c5b63ca6b5d611dd18fd92cf3e9f"
server_key_iv: "81293244c2ea60f594c9121e47853c7a"
server_key_version: "encryptionkey@0"
I20260807 08:47:56.417573 12642 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786092453486433-4734-0/minicluster-data/master-0/wal/instance:
uuid: "93ba8fc84a3347eb8bd89f6f85efcedd"
format_stamp: "Formatted at 2026-08-07 08:47:56 on dist-test-slave-6zbq"
server_key: "4102c5b63ca6b5d611dd18fd92cf3e9f"
server_key_iv: "81293244c2ea60f594c9121e47853c7a"
server_key_version: "encryptionkey@0"
I20260807 08:47:56.418815 12642 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 08:47:56.419441 12661 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:56.419596 12642 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 08:47:56.419678 12642 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786092453486433-4734-0/minicluster-data/master-0/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786092453486433-4734-0/minicluster-data/master-0/wal
uuid: "93ba8fc84a3347eb8bd89f6f85efcedd"
format_stamp: "Formatted at 2026-08-07 08:47:56 on dist-test-slave-6zbq"
server_key: "4102c5b63ca6b5d611dd18fd92cf3e9f"
server_key_iv: "81293244c2ea60f594c9121e47853c7a"
server_key_version: "encryptionkey@0"
I20260807 08:47:56.419741 12642 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786092453486433-4734-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786092453486433-4734-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786092453486433-4734-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:47:56.440624 12642 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:47:56.440887 12642 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:47:56.441010 12642 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:47:56.444808 12642 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.190:46767
I20260807 08:47:56.444841 12740 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.190:46767 every 8 connection(s)
I20260807 08:47:56.445143 12642 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
I20260807 08:47:56.446017 12742 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:47:56.448302 12742 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 93ba8fc84a3347eb8bd89f6f85efcedd: Bootstrap starting.
I20260807 08:47:56.448850 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 12642
I20260807 08:47:56.448899 12742 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 93ba8fc84a3347eb8bd89f6f85efcedd: Neither blocks nor log segments found. Creating new log.
I20260807 08:47:56.448966 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786092453486433-4734-0/minicluster-data/master-0/wal/instance
I20260807 08:47:56.449151 4734 external_mini_cluster.cc:1428] Setting key 6b28ef9c168c9ffc3bf732d7b8e514b5
I20260807 08:47:56.449221 12742 log.cc:824] T 00000000000000000000000000000000 P 93ba8fc84a3347eb8bd89f6f85efcedd: Log is configured to *not* fsync() on all Append() calls
I20260807 08:47:56.450060 12742 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 93ba8fc84a3347eb8bd89f6f85efcedd: No bootstrap required, opened a new log
I20260807 08:47:56.452498 12742 raft_consensus.cc:348] T 00000000000000000000000000000000 P 93ba8fc84a3347eb8bd89f6f85efcedd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93ba8fc84a3347eb8bd89f6f85efcedd" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 46767 } }
I20260807 08:47:56.452629 12742 raft_consensus.cc:374] T 00000000000000000000000000000000 P 93ba8fc84a3347eb8bd89f6f85efcedd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:47:56.452649 12742 raft_consensus.cc:729] T 00000000000000000000000000000000 P 93ba8fc84a3347eb8bd89f6f85efcedd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93ba8fc84a3347eb8bd89f6f85efcedd, State: Initialized, Role: FOLLOWER
I20260807 08:47:56.452759 12742 consensus_queue.cc:260] T 00000000000000000000000000000000 P 93ba8fc84a3347eb8bd89f6f85efcedd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93ba8fc84a3347eb8bd89f6f85efcedd" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 46767 } }
I20260807 08:47:56.452827 12742 raft_consensus.cc:388] T 00000000000000000000000000000000 P 93ba8fc84a3347eb8bd89f6f85efcedd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:47:56.452848 12742 raft_consensus.cc:482] T 00000000000000000000000000000000 P 93ba8fc84a3347eb8bd89f6f85efcedd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:47:56.452894 12742 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 93ba8fc84a3347eb8bd89f6f85efcedd [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:47:56.453542 12742 raft_consensus.cc:504] T 00000000000000000000000000000000 P 93ba8fc84a3347eb8bd89f6f85efcedd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93ba8fc84a3347eb8bd89f6f85efcedd" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 46767 } }
I20260807 08:47:56.453651 12742 leader_election.cc:302] T 00000000000000000000000000000000 P 93ba8fc84a3347eb8bd89f6f85efcedd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 93ba8fc84a3347eb8bd89f6f85efcedd; no voters:
I20260807 08:47:56.453814 12742 leader_election.cc:288] T 00000000000000000000000000000000 P 93ba8fc84a3347eb8bd89f6f85efcedd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:47:56.454003 12748 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 93ba8fc84a3347eb8bd89f6f85efcedd [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:47:56.454056 12742 sys_catalog.cc:564] T 00000000000000000000000000000000 P 93ba8fc84a3347eb8bd89f6f85efcedd [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:47:56.454569 12748 raft_consensus.cc:686] T 00000000000000000000000000000000 P 93ba8fc84a3347eb8bd89f6f85efcedd [term 1 LEADER]: Becoming Leader. State: Replica: 93ba8fc84a3347eb8bd89f6f85efcedd, State: Running, Role: LEADER
I20260807 08:47:56.454679 12748 consensus_queue.cc:237] T 00000000000000000000000000000000 P 93ba8fc84a3347eb8bd89f6f85efcedd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93ba8fc84a3347eb8bd89f6f85efcedd" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 46767 } }
I20260807 08:47:56.455111 12748 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93ba8fc84a3347eb8bd89f6f85efcedd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "93ba8fc84a3347eb8bd89f6f85efcedd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93ba8fc84a3347eb8bd89f6f85efcedd" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 46767 } } }
W20260807 08:47:56.455906 12770 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 93ba8fc84a3347eb8bd89f6f85efcedd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 08:47:56.455962 12770 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 08:47:56.456139 12749 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93ba8fc84a3347eb8bd89f6f85efcedd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 93ba8fc84a3347eb8bd89f6f85efcedd. Latest consensus state: current_term: 1 leader_uuid: "93ba8fc84a3347eb8bd89f6f85efcedd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93ba8fc84a3347eb8bd89f6f85efcedd" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 46767 } } }
I20260807 08:47:56.456153 12747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:56.449634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59508 (local address 127.4.159.190:46767)
0807 08:47:56.449812 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:56.449820 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:56.449853 (+ 33us) server_negotiation.cc:407] Connection header received
0807 08:47:56.450442 (+ 589us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:56.450451 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:56.450528 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:56.450683 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:47:56.450832 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:56.452172 (+ 1340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:56.453070 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:56.453678 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:56.453802 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:56.454891 (+ 1089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:47:56.454895 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:47:56.454896 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:47:56.455473 (+ 577us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:47:56.455520 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:47:56.455535 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:47:56.455602 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:56.455682 (+ 80us) server_negotiation.cc:299] Negotiation successful
0807 08:47:56.455877 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":62,"threads_started":1}
I20260807 08:47:56.456241 12749 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93ba8fc84a3347eb8bd89f6f85efcedd [sys.catalog]: This master's current role is: LEADER
I20260807 08:47:56.455264 12748 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93ba8fc84a3347eb8bd89f6f85efcedd [sys.catalog]: This master's current role is: LEADER
I20260807 08:47:56.457068 12772 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:47:56.457119 4734 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 08:47:56.457283 12772 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:47:56.458779 12772 catalog_manager.cc:1347] Generated new cluster ID: c841896fb42e4f44b6109c863df77eed
I20260807 08:47:56.458822 12772 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:47:56.471637 12772 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:47:56.472260 12772 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:47:56.490538 12772 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 93ba8fc84a3347eb8bd89f6f85efcedd: Generated new TSK 0
I20260807 08:47:56.490705 12772 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:47:56.519615 12747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:56.502701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59510 (local address 127.4.159.190:46767)
0807 08:47:56.502749 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:56.502751 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:56.502845 (+ 94us) server_negotiation.cc:407] Connection header received
0807 08:47:56.502928 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:56.502929 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:56.502939 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:56.502998 (+ 59us) server_negotiation.cc:226] Negotiated authn=JWT
0807 08:47:56.503155 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:56.503644 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:56.504754 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:56.504932 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:56.504971 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:56.505139 (+ 168us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 08:47:56.519411 (+ 14272us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0807 08:47:56.519555 (+ 144us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:59510: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":25}
W20260807 08:47:56.519680 12747 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:59510: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260807 08:47:56.519806 12784 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:47:56.502618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:46767 (local address 127.0.0.1:59510)
0807 08:47:56.502774 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0807 08:47:56.502785 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 08:47:56.502857 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:47:56.503009 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:47:56.503010 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:47:56.503012 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0807 08:47:56.503071 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:56.503073 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:56.503747 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:56.503748 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:56.504673 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:56.504676 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:56.504988 (+ 312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:56.504989 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:56.505057 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:56.505062 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 08:47:56.519546 (+ 14484us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0807 08:47:56.519687 (+ 141us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.4.159.190:46767: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":86,"threads_started":1}
I20260807 08:47:56.520103 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 12642
2026-08-07T08:47:56Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (230 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (91 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (88 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786092453486433-4734-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:47:56 dist-test-slave-6zbq krb5kdc[12801](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:47:56 dist-test-slave-6zbq krb5kdc[12801](info): set up 2 sockets
Aug 07 08:47:56 dist-test-slave-6zbq krb5kdc[12801](info): commencing operation
krb5kdc: starting...
W20260807 08:47:57.834666 4734 mini_kdc.cc:121] Time spent starting KDC: real 1.107s user 0.006s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786092453486433-4734-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786092453486433-4734-0/krb5kdc/test-user.keytab.
Aug 07 08:47:57 dist-test-slave-6zbq krb5kdc[12801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092477, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:47:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:47:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.159.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.159.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.159.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab.
Entry for principal kudu/127.4.159.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab.
I20260807 08:47:57.949633 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786092453486433-4734-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786092453486433-4734-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786092453486433-4734-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.159.190:38689
--webserver_interface=127.4.159.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab
--principal=kudu/127.4.159.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:34065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.159.190:38689
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:47:58.032658 12820 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:47:58.032857 12820 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:47:58.032878 12820 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:47:58.033967 12820 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:47:58.034004 12820 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:47:58.034017 12820 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:47:58.034029 12820 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:47:58.034039 12820 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:47:58.035475 12820 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:34065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786092453486433-4734-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786092453486433-4734-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.159.190:38689
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab
--principal=kudu/127.4.159.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.159.190:38689
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.159.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12820
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786092453486433-4734-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:47:58.035688 12820 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:47:58.035871 12820 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:47:58.040620 12827 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:47:58.040655 12826 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:47:58.040611 12830 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:47:58.040815 12820 server_base.cc:1034] running on GCE node
I20260807 08:47:58.041042 12820 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:47:58.041316 12820 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:47:58.042448 12820 hybrid_clock.cc:648] HybridClock initialized: now 1786092478042441 us; error 28 us; skew 500 ppm
Aug 07 08:47:58 dist-test-slave-6zbq krb5kdc[12801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092478, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:47:58.044889 12820 init.cc:377] Logged in from keytab as kudu/127.4.159.190@KRBTEST.COM (short username kudu)
I20260807 08:47:58.045295 12820 webserver.cc:460] Webserver started at http://127.4.159.190:32965/ using document root <none> and password file <none>
I20260807 08:47:58.045499 12820 fs_manager.cc:359] Metadata directory not provided
I20260807 08:47:58.045549 12820 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:47:58.045657 12820 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:47:58.046898 12820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786092453486433-4734-0/minicluster-data/master-0/data/instance:
uuid: "e66cdaa17e4e4ff38b0b5d67a0f63d91"
format_stamp: "Formatted at 2026-08-07 08:47:58 on dist-test-slave-6zbq"
server_key: "fcf7b3e9c533b231bd4e658c11ea965d"
server_key_iv: "a27daad2c5f636deec1049a7c89b7f52"
server_key_version: "encryptionkey@0"
I20260807 08:47:58.047240 12820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786092453486433-4734-0/minicluster-data/master-0/wal/instance:
uuid: "e66cdaa17e4e4ff38b0b5d67a0f63d91"
format_stamp: "Formatted at 2026-08-07 08:47:58 on dist-test-slave-6zbq"
server_key: "fcf7b3e9c533b231bd4e658c11ea965d"
server_key_iv: "a27daad2c5f636deec1049a7c89b7f52"
server_key_version: "encryptionkey@0"
I20260807 08:47:58.048491 12820 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 08:47:58.049142 12839 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:58.049284 12820 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 08:47:58.049369 12820 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786092453486433-4734-0/minicluster-data/master-0/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786092453486433-4734-0/minicluster-data/master-0/wal
uuid: "e66cdaa17e4e4ff38b0b5d67a0f63d91"
format_stamp: "Formatted at 2026-08-07 08:47:58 on dist-test-slave-6zbq"
server_key: "fcf7b3e9c533b231bd4e658c11ea965d"
server_key_iv: "a27daad2c5f636deec1049a7c89b7f52"
server_key_version: "encryptionkey@0"
I20260807 08:47:58.049431 12820 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786092453486433-4734-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786092453486433-4734-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786092453486433-4734-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:47:58.093055 12820 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:47:58.094856 12820 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:47:58.094966 12820 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:47:58.098762 12820 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.190:38689
I20260807 08:47:58.098815 12919 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.190:38689 every 8 connection(s)
I20260807 08:47:58.099131 12820 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
I20260807 08:47:58.099969 12920 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:47:58.102210 12920 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e66cdaa17e4e4ff38b0b5d67a0f63d91: Bootstrap starting.
I20260807 08:47:58.102800 12920 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e66cdaa17e4e4ff38b0b5d67a0f63d91: Neither blocks nor log segments found. Creating new log.
I20260807 08:47:58.103029 12920 log.cc:824] T 00000000000000000000000000000000 P e66cdaa17e4e4ff38b0b5d67a0f63d91: Log is configured to *not* fsync() on all Append() calls
I20260807 08:47:58.103683 12920 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e66cdaa17e4e4ff38b0b5d67a0f63d91: No bootstrap required, opened a new log
I20260807 08:47:58.104430 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 12820
I20260807 08:47:58.104542 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786092453486433-4734-0/minicluster-data/master-0/wal/instance
I20260807 08:47:58.104750 4734 external_mini_cluster.cc:1428] Setting key d6dd99c3ef19981b97644fa63bc0bc77
I20260807 08:47:58.105813 12920 raft_consensus.cc:348] T 00000000000000000000000000000000 P e66cdaa17e4e4ff38b0b5d67a0f63d91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e66cdaa17e4e4ff38b0b5d67a0f63d91" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 38689 } }
I20260807 08:47:58.105995 12920 raft_consensus.cc:374] T 00000000000000000000000000000000 P e66cdaa17e4e4ff38b0b5d67a0f63d91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:47:58.106019 12920 raft_consensus.cc:729] T 00000000000000000000000000000000 P e66cdaa17e4e4ff38b0b5d67a0f63d91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e66cdaa17e4e4ff38b0b5d67a0f63d91, State: Initialized, Role: FOLLOWER
I20260807 08:47:58.106129 12920 consensus_queue.cc:260] T 00000000000000000000000000000000 P e66cdaa17e4e4ff38b0b5d67a0f63d91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e66cdaa17e4e4ff38b0b5d67a0f63d91" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 38689 } }
I20260807 08:47:58.106222 12920 raft_consensus.cc:388] T 00000000000000000000000000000000 P e66cdaa17e4e4ff38b0b5d67a0f63d91 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:47:58.106267 12920 raft_consensus.cc:482] T 00000000000000000000000000000000 P e66cdaa17e4e4ff38b0b5d67a0f63d91 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:47:58.106317 12920 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e66cdaa17e4e4ff38b0b5d67a0f63d91 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:47:58.107347 12920 raft_consensus.cc:504] T 00000000000000000000000000000000 P e66cdaa17e4e4ff38b0b5d67a0f63d91 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e66cdaa17e4e4ff38b0b5d67a0f63d91" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 38689 } }
I20260807 08:47:58.107440 12920 leader_election.cc:302] T 00000000000000000000000000000000 P e66cdaa17e4e4ff38b0b5d67a0f63d91 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e66cdaa17e4e4ff38b0b5d67a0f63d91; no voters:
I20260807 08:47:58.107585 12920 leader_election.cc:288] T 00000000000000000000000000000000 P e66cdaa17e4e4ff38b0b5d67a0f63d91 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:47:58.107633 12932 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e66cdaa17e4e4ff38b0b5d67a0f63d91 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:47:58.107776 12932 raft_consensus.cc:686] T 00000000000000000000000000000000 P e66cdaa17e4e4ff38b0b5d67a0f63d91 [term 1 LEADER]: Becoming Leader. State: Replica: e66cdaa17e4e4ff38b0b5d67a0f63d91, State: Running, Role: LEADER
I20260807 08:47:58.107826 12920 sys_catalog.cc:564] T 00000000000000000000000000000000 P e66cdaa17e4e4ff38b0b5d67a0f63d91 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:47:58.107861 12932 consensus_queue.cc:237] T 00000000000000000000000000000000 P e66cdaa17e4e4ff38b0b5d67a0f63d91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e66cdaa17e4e4ff38b0b5d67a0f63d91" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 38689 } }
I20260807 08:47:58.108278 12935 sys_catalog.cc:455] T 00000000000000000000000000000000 P e66cdaa17e4e4ff38b0b5d67a0f63d91 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e66cdaa17e4e4ff38b0b5d67a0f63d91. Latest consensus state: current_term: 1 leader_uuid: "e66cdaa17e4e4ff38b0b5d67a0f63d91" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e66cdaa17e4e4ff38b0b5d67a0f63d91" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 38689 } } }
I20260807 08:47:58.108386 12935 sys_catalog.cc:458] T 00000000000000000000000000000000 P e66cdaa17e4e4ff38b0b5d67a0f63d91 [sys.catalog]: This master's current role is: LEADER
I20260807 08:47:58.108287 12934 sys_catalog.cc:455] T 00000000000000000000000000000000 P e66cdaa17e4e4ff38b0b5d67a0f63d91 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e66cdaa17e4e4ff38b0b5d67a0f63d91" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e66cdaa17e4e4ff38b0b5d67a0f63d91" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 38689 } } }
I20260807 08:47:58.108588 12934 sys_catalog.cc:458] T 00000000000000000000000000000000 P e66cdaa17e4e4ff38b0b5d67a0f63d91 [sys.catalog]: This master's current role is: LEADER
I20260807 08:47:58.108706 12945 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:47:58.109436 12945 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:47:58.110971 12945 catalog_manager.cc:1347] Generated new cluster ID: 6c6196bfa6ac450db1ed3f1adc11e840
I20260807 08:47:58.111021 12945 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 08:47:58 dist-test-slave-6zbq krb5kdc[12801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092477, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.159.190@KRBTEST.COM
I20260807 08:47:58.116240 12928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:58.105627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42806 (local address 127.4.159.190:38689)
0807 08:47:58.105819 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:58.105826 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:58.105858 (+ 32us) server_negotiation.cc:407] Connection header received
0807 08:47:58.105963 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:58.105969 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:58.106026 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:58.106161 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:47:58.106981 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:58.108177 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:58.109226 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:58.109964 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:58.110074 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:58.112766 (+ 2692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:47:58.112770 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:47:58.112771 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:47:58.112783 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:47:58.114662 (+ 1879us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:58.115018 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:58.115021 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:58.115021 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:58.115166 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:58.115350 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:58.115351 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:58.115351 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:58.115672 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:47:58.115829 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:58.115896 (+ 67us) server_negotiation.cc:299] Negotiation successful
0807 08:47:58.116033 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":97,"threads_started":1}
I20260807 08:47:58.128111 12945 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:47:58.128844 12945 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:47:58.136556 12945 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e66cdaa17e4e4ff38b0b5d67a0f63d91: Generated new TSK 0
I20260807 08:47:58.136734 12945 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.159.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.159.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.159.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.129.keytab.
Entry for principal kudu/127.4.159.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.129.keytab.
I20260807 08:47:58.162892 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786092453486433-4734-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786092453486433-4734-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786092453486433-4734-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786092453486433-4734-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.159.129:0
--local_ip_for_outbound_sockets=127.4.159.129
--webserver_interface=127.4.159.129
--webserver_port=0
--tserver_master_addrs=127.4.159.190:38689
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.129.keytab
--principal=kudu/127.4.159.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:34065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:47:58.242066 12964 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:47:58.242254 12964 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:47:58.242282 12964 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:47:58.243412 12964 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:47:58.243453 12964 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:47:58.243486 12964 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.159.129
I20260807 08:47:58.244928 12964 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:34065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786092453486433-4734-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786092453486433-4734-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.129.keytab
--principal=kudu/127.4.159.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.159.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786092453486433-4734-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.159.129
--webserver_port=0
--tserver_master_addrs=127.4.159.190:38689
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12964
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.159.129
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786092453486433-4734-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:47:58.245146 12964 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:47:58.245337 12964 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:47:58.250027 12975 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:47:58.250056 12969 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:47:58.250165 12964 server_base.cc:1034] running on GCE node
W20260807 08:47:58.250051 12971 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:47:58.250519 12964 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:47:58.250772 12964 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:47:58.251940 12964 hybrid_clock.cc:648] HybridClock initialized: now 1786092478251926 us; error 29 us; skew 500 ppm
Aug 07 08:47:58 dist-test-slave-6zbq krb5kdc[12801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092478, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:47:58.254281 12964 init.cc:377] Logged in from keytab as kudu/127.4.159.129@KRBTEST.COM (short username kudu)
I20260807 08:47:58.254719 12964 webserver.cc:460] Webserver started at http://127.4.159.129:45903/ using document root <none> and password file <none>
I20260807 08:47:58.254904 12964 fs_manager.cc:359] Metadata directory not provided
I20260807 08:47:58.254945 12964 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:47:58.255064 12964 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:47:58.256021 12964 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786092453486433-4734-0/minicluster-data/ts-0/data/instance:
uuid: "fa2fd0b34579455686a9727a92f2aa72"
format_stamp: "Formatted at 2026-08-07 08:47:58 on dist-test-slave-6zbq"
server_key: "f1050433477ca49074c6d0ad75c021a5"
server_key_iv: "fef8d4da3a8bbd1fed98f17d3a2e8446"
server_key_version: "encryptionkey@0"
I20260807 08:47:58.256332 12964 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786092453486433-4734-0/minicluster-data/ts-0/wal/instance:
uuid: "fa2fd0b34579455686a9727a92f2aa72"
format_stamp: "Formatted at 2026-08-07 08:47:58 on dist-test-slave-6zbq"
server_key: "f1050433477ca49074c6d0ad75c021a5"
server_key_iv: "fef8d4da3a8bbd1fed98f17d3a2e8446"
server_key_version: "encryptionkey@0"
I20260807 08:47:58.257519 12964 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 08:47:58.258194 12982 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:58.258394 12964 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 08:47:58.258476 12964 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786092453486433-4734-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786092453486433-4734-0/minicluster-data/ts-0/wal
uuid: "fa2fd0b34579455686a9727a92f2aa72"
format_stamp: "Formatted at 2026-08-07 08:47:58 on dist-test-slave-6zbq"
server_key: "f1050433477ca49074c6d0ad75c021a5"
server_key_iv: "fef8d4da3a8bbd1fed98f17d3a2e8446"
server_key_version: "encryptionkey@0"
I20260807 08:47:58.258541 12964 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786092453486433-4734-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786092453486433-4734-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786092453486433-4734-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:47:58.283496 12964 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:47:58.285430 12964 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:47:58.285567 12964 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:47:58.285774 12964 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:47:58.286041 12964 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:47:58.286072 12964 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:58.286092 12964 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:47:58.286103 12964 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:58.291450 12964 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.129:39179
I20260807 08:47:58.291494 13148 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.129:39179 every 8 connection(s)
I20260807 08:47:58.291775 12964 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786092453486433-4734-0/minicluster-data/ts-0/data/info.pb
Aug 07 08:47:58 dist-test-slave-6zbq krb5kdc[12801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092478, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.129@KRBTEST.COM for kudu/127.4.159.190@KRBTEST.COM
I20260807 08:47:58.297739 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 12964
I20260807 08:47:58.297837 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786092453486433-4734-0/minicluster-data/ts-0/wal/instance
I20260807 08:47:58.298038 4734 external_mini_cluster.cc:1428] Setting key db2f2e196d568eba5eecfa875fea0b8f
I20260807 08:47:58.300510 12928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:58.292429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.129:55377 (local address 127.4.159.190:38689)
0807 08:47:58.292501 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:58.292504 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:58.292631 (+ 127us) server_negotiation.cc:407] Connection header received
0807 08:47:58.293485 (+ 854us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:58.293486 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:58.293499 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:58.293548 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:47:58.294100 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:58.294681 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:58.295600 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:58.295796 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:58.295843 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:58.297752 (+ 1909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:47:58.297754 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:47:58.297755 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:47:58.297764 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:47:58.299328 (+ 1564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:58.299676 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:58.299677 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:58.299677 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:58.299738 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:58.299928 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:58.299929 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:58.299929 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:58.300052 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:47:58.300108 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:58.300337 (+ 229us) server_negotiation.cc:299] Negotiation successful
0807 08:47:58.300424 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 08:47:58.300753 13155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:58.292390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:38689 (local address 127.4.159.129:55377)
0807 08:47:58.292525 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0807 08:47:58.292560 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0807 08:47:58.293407 (+ 847us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:47:58.293600 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:47:58.293603 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:47:58.293838 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:47:58.294017 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:58.294021 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:58.294765 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:58.294766 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:58.295524 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:58.295526 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:58.295851 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:58.295852 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:58.295944 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:58.296374 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:47:58.296383 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:47:58.297656 (+ 1273us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:47:58.299402 (+ 1746us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:47:58.299408 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:47:58.299410 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:47:58.299611 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:47:58.299795 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:47:58.299795 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:47:58.299796 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:47:58.299883 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:47:58.300106 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:47:58.300108 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:47:58.300211 (+ 103us) client_negotiation.cc:769] Sending connection context
0807 08:47:58.300310 (+ 99us) client_negotiation.cc:240] Negotiation successful
0807 08:47:58.300470 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":73,"threads_started":1}
I20260807 08:47:58.301215 13152 heartbeater.cc:347] Connected to a master server at 127.4.159.190:38689
I20260807 08:47:58.301301 13152 heartbeater.cc:464] Registering TS with master...
I20260807 08:47:58.301460 13152 heartbeater.cc:511] Master 127.4.159.190:38689 requested a full tablet report, sending...
I20260807 08:47:58.302008 12870 ts_manager.cc:194] Registered new tserver with Master: fa2fd0b34579455686a9727a92f2aa72 (127.4.159.129:39179)
I20260807 08:47:58.302817 12870 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.159.129@KRBTEST.COM'} at 127.4.159.129:55377
WARNING: no policy specified for kudu/127.4.159.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.159.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.159.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.130.keytab.
Entry for principal kudu/127.4.159.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.130.keytab.
I20260807 08:47:58.320602 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786092453486433-4734-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786092453486433-4734-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786092453486433-4734-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786092453486433-4734-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.159.130:0
--local_ip_for_outbound_sockets=127.4.159.130
--webserver_interface=127.4.159.130
--webserver_port=0
--tserver_master_addrs=127.4.159.190:38689
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.130.keytab
--principal=kudu/127.4.159.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:34065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:47:58.400127 13160 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:47:58.400303 13160 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:47:58.400326 13160 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:47:58.401458 13160 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:47:58.401503 13160 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:47:58.401532 13160 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.159.130
I20260807 08:47:58.402990 13160 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:34065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786092453486433-4734-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786092453486433-4734-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.130.keytab
--principal=kudu/127.4.159.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.159.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786092453486433-4734-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.159.130
--webserver_port=0
--tserver_master_addrs=127.4.159.190:38689
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13160
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.159.130
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786092453486433-4734-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:47:58.403198 13160 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:47:58.403389 13160 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:47:58.407975 13168 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:47:58.407989 13174 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:47:58.408037 13160 server_base.cc:1034] running on GCE node
W20260807 08:47:58.407979 13170 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:47:58.408360 13160 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:47:58.408540 13160 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:47:58.409677 13160 hybrid_clock.cc:648] HybridClock initialized: now 1786092478409667 us; error 28 us; skew 500 ppm
Aug 07 08:47:58 dist-test-slave-6zbq krb5kdc[12801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092478, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:47:58.411921 13160 init.cc:377] Logged in from keytab as kudu/127.4.159.130@KRBTEST.COM (short username kudu)
I20260807 08:47:58.412313 13160 webserver.cc:460] Webserver started at http://127.4.159.130:38443/ using document root <none> and password file <none>
I20260807 08:47:58.412515 13160 fs_manager.cc:359] Metadata directory not provided
I20260807 08:47:58.412557 13160 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:47:58.412688 13160 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:47:58.413625 13160 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786092453486433-4734-0/minicluster-data/ts-1/data/instance:
uuid: "978b0975e32b41b983e24e7e9a0480f0"
format_stamp: "Formatted at 2026-08-07 08:47:58 on dist-test-slave-6zbq"
server_key: "a91ab5c34a2c3fd842189e6a9d633363"
server_key_iv: "7bb3304154cb3a44030817226178ade4"
server_key_version: "encryptionkey@0"
I20260807 08:47:58.413931 13160 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786092453486433-4734-0/minicluster-data/ts-1/wal/instance:
uuid: "978b0975e32b41b983e24e7e9a0480f0"
format_stamp: "Formatted at 2026-08-07 08:47:58 on dist-test-slave-6zbq"
server_key: "a91ab5c34a2c3fd842189e6a9d633363"
server_key_iv: "7bb3304154cb3a44030817226178ade4"
server_key_version: "encryptionkey@0"
I20260807 08:47:58.415155 13160 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 08:47:58.415777 13183 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:58.415913 13160 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 08:47:58.415977 13160 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786092453486433-4734-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786092453486433-4734-0/minicluster-data/ts-1/wal
uuid: "978b0975e32b41b983e24e7e9a0480f0"
format_stamp: "Formatted at 2026-08-07 08:47:58 on dist-test-slave-6zbq"
server_key: "a91ab5c34a2c3fd842189e6a9d633363"
server_key_iv: "7bb3304154cb3a44030817226178ade4"
server_key_version: "encryptionkey@0"
I20260807 08:47:58.416038 13160 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786092453486433-4734-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786092453486433-4734-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786092453486433-4734-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:47:58.430934 13160 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:47:58.432435 13160 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:47:58.432552 13160 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:47:58.432746 13160 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:47:58.433005 13160 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:47:58.433036 13160 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:58.433056 13160 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:47:58.433079 13160 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:58.438421 13160 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.130:35675
I20260807 08:47:58.438479 13348 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.130:35675 every 8 connection(s)
I20260807 08:47:58.438746 13160 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786092453486433-4734-0/minicluster-data/ts-1/data/info.pb
Aug 07 08:47:58 dist-test-slave-6zbq krb5kdc[12801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092478, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.130@KRBTEST.COM for kudu/127.4.159.190@KRBTEST.COM
I20260807 08:47:58.444857 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 13160
I20260807 08:47:58.444963 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786092453486433-4734-0/minicluster-data/ts-1/wal/instance
I20260807 08:47:58.445183 4734 external_mini_cluster.cc:1428] Setting key 83309fe9600615f26832b440b7491949
I20260807 08:47:58.447494 12928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:58.439381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.130:55901 (local address 127.4.159.190:38689)
0807 08:47:58.439435 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:58.439438 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:58.439642 (+ 204us) server_negotiation.cc:407] Connection header received
0807 08:47:58.440548 (+ 906us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:58.440550 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:58.440561 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:58.440620 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:47:58.441224 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:58.441719 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:58.442641 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:58.442840 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:58.442898 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:58.444771 (+ 1873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:47:58.444773 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:47:58.444773 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:47:58.444781 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:47:58.446129 (+ 1348us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:58.446597 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:58.446598 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:58.446599 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:58.446671 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:58.446918 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:58.446919 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:58.446919 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:58.447021 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:47:58.447068 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:58.447341 (+ 273us) server_negotiation.cc:299] Negotiation successful
0807 08:47:58.447407 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 08:47:58.447793 13351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:58.439353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:38689 (local address 127.4.159.130:55901)
0807 08:47:58.439510 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0807 08:47:58.439544 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0807 08:47:58.440435 (+ 891us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:47:58.440687 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:47:58.440690 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:47:58.440923 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:47:58.441114 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:58.441119 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:58.441791 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:58.441791 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:58.442573 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:58.442575 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:58.442903 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:58.442904 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:58.442991 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:58.443398 (+ 407us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:47:58.443405 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:47:58.444688 (+ 1283us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:47:58.446201 (+ 1513us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:47:58.446207 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:47:58.446210 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:47:58.446515 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:47:58.446734 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:47:58.446735 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:47:58.446735 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:47:58.446858 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:47:58.447073 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:47:58.447078 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:47:58.447223 (+ 145us) client_negotiation.cc:769] Sending connection context
0807 08:47:58.447327 (+ 104us) client_negotiation.cc:240] Negotiation successful
0807 08:47:58.447486 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":77,"threads_started":1}
I20260807 08:47:58.448227 13349 heartbeater.cc:347] Connected to a master server at 127.4.159.190:38689
I20260807 08:47:58.448310 13349 heartbeater.cc:464] Registering TS with master...
I20260807 08:47:58.448470 13349 heartbeater.cc:511] Master 127.4.159.190:38689 requested a full tablet report, sending...
I20260807 08:47:58.448832 12870 ts_manager.cc:194] Registered new tserver with Master: 978b0975e32b41b983e24e7e9a0480f0 (127.4.159.130:35675)
I20260807 08:47:58.449184 12870 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.159.130@KRBTEST.COM'} at 127.4.159.130:55901
WARNING: no policy specified for kudu/127.4.159.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.159.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.159.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.131.keytab.
Entry for principal kudu/127.4.159.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.131.keytab.
I20260807 08:47:58.468223 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786092453486433-4734-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786092453486433-4734-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786092453486433-4734-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786092453486433-4734-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.159.131:0
--local_ip_for_outbound_sockets=127.4.159.131
--webserver_interface=127.4.159.131
--webserver_port=0
--tserver_master_addrs=127.4.159.190:38689
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.131.keytab
--principal=kudu/127.4.159.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:34065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:47:58.546990 13363 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:47:58.547169 13363 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:47:58.547190 13363 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:47:58.548257 13363 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:47:58.548290 13363 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:47:58.548312 13363 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.159.131
I20260807 08:47:58.549714 13363 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:34065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786092453486433-4734-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786092453486433-4734-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.131.keytab
--principal=kudu/127.4.159.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.159.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786092453486433-4734-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.159.131
--webserver_port=0
--tserver_master_addrs=127.4.159.190:38689
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13363
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.159.131
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786092453486433-4734-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:47:58.549888 13363 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:47:58.550055 13363 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:47:58.554616 13369 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:47:58.554596 13374 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:47:58.554615 13370 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:47:58.554939 13363 server_base.cc:1034] running on GCE node
I20260807 08:47:58.555122 13363 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:47:58.555341 13363 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:47:58.556480 13363 hybrid_clock.cc:648] HybridClock initialized: now 1786092478556462 us; error 28 us; skew 500 ppm
Aug 07 08:47:58 dist-test-slave-6zbq krb5kdc[12801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092478, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:47:58.558868 13363 init.cc:377] Logged in from keytab as kudu/127.4.159.131@KRBTEST.COM (short username kudu)
I20260807 08:47:58.559314 13363 webserver.cc:460] Webserver started at http://127.4.159.131:34201/ using document root <none> and password file <none>
I20260807 08:47:58.559566 13363 fs_manager.cc:359] Metadata directory not provided
I20260807 08:47:58.559616 13363 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:47:58.559742 13363 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:47:58.560750 13363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786092453486433-4734-0/minicluster-data/ts-2/data/instance:
uuid: "b4b8fd6650c7435ab1c390e4921705cf"
format_stamp: "Formatted at 2026-08-07 08:47:58 on dist-test-slave-6zbq"
server_key: "aaf5436f1ac232d1b83323f9efa266be"
server_key_iv: "45aba8eefb7f52da15f01fe0952f83fd"
server_key_version: "encryptionkey@0"
I20260807 08:47:58.561067 13363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786092453486433-4734-0/minicluster-data/ts-2/wal/instance:
uuid: "b4b8fd6650c7435ab1c390e4921705cf"
format_stamp: "Formatted at 2026-08-07 08:47:58 on dist-test-slave-6zbq"
server_key: "aaf5436f1ac232d1b83323f9efa266be"
server_key_iv: "45aba8eefb7f52da15f01fe0952f83fd"
server_key_version: "encryptionkey@0"
I20260807 08:47:58.562494 13363 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260807 08:47:58.563344 13382 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:58.563524 13363 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:47:58.563611 13363 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786092453486433-4734-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786092453486433-4734-0/minicluster-data/ts-2/wal
uuid: "b4b8fd6650c7435ab1c390e4921705cf"
format_stamp: "Formatted at 2026-08-07 08:47:58 on dist-test-slave-6zbq"
server_key: "aaf5436f1ac232d1b83323f9efa266be"
server_key_iv: "45aba8eefb7f52da15f01fe0952f83fd"
server_key_version: "encryptionkey@0"
I20260807 08:47:58.563688 13363 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786092453486433-4734-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786092453486433-4734-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786092453486433-4734-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:47:58.578054 13363 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:47:58.579769 13363 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:47:58.579900 13363 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:47:58.580098 13363 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:47:58.580376 13363 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:47:58.580410 13363 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:58.580437 13363 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:47:58.580456 13363 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:47:58.585901 13363 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.131:42325
I20260807 08:47:58.585938 13553 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.131:42325 every 8 connection(s)
I20260807 08:47:58.586210 13363 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786092453486433-4734-0/minicluster-data/ts-2/data/info.pb
Aug 07 08:47:58 dist-test-slave-6zbq krb5kdc[12801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092478, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.131@KRBTEST.COM for kudu/127.4.159.190@KRBTEST.COM
I20260807 08:47:58.592634 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 13363
I20260807 08:47:58.592742 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786092453486433-4734-0/minicluster-data/ts-2/wal/instance
I20260807 08:47:58.592916 4734 external_mini_cluster.cc:1428] Setting key 80df694530e818fb921909d3c5884c94
I20260807 08:47:58.594496 12928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:58.586965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.131:39273 (local address 127.4.159.190:38689)
0807 08:47:58.587037 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:58.587039 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:58.587180 (+ 141us) server_negotiation.cc:407] Connection header received
0807 08:47:58.588011 (+ 831us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:58.588013 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:58.588023 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:58.588071 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:47:58.588579 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:58.589064 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:58.589965 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:58.590138 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:58.590178 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:58.592050 (+ 1872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:47:58.592052 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:47:58.592052 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:47:58.592058 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:47:58.593333 (+ 1275us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:58.593738 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:58.593740 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:58.593740 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:58.593805 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:58.594010 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:58.594010 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:58.594010 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:58.594107 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:47:58.594154 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:58.594356 (+ 202us) server_negotiation.cc:299] Negotiation successful
0807 08:47:58.594434 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 08:47:58.594668 13558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:58.586931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:38689 (local address 127.4.159.131:39273)
0807 08:47:58.587062 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0807 08:47:58.587096 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0807 08:47:58.587934 (+ 838us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:47:58.588125 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:47:58.588128 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:47:58.588348 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:47:58.588516 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:58.588520 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:58.589124 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:58.589125 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:58.589907 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:47:58.589909 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:47:58.590187 (+ 278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:47:58.590188 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:47:58.590280 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:58.590705 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:47:58.590712 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:47:58.591969 (+ 1257us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:47:58.593400 (+ 1431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:47:58.593407 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:47:58.593409 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:47:58.593674 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:47:58.593857 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:47:58.593858 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:47:58.593859 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:47:58.593960 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:47:58.594149 (+ 189us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:47:58.594152 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:47:58.594250 (+ 98us) client_negotiation.cc:769] Sending connection context
0807 08:47:58.594322 (+ 72us) client_negotiation.cc:240] Negotiation successful
0807 08:47:58.594478 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":70,"threads_started":1}
I20260807 08:47:58.595046 13554 heartbeater.cc:347] Connected to a master server at 127.4.159.190:38689
I20260807 08:47:58.595124 13554 heartbeater.cc:464] Registering TS with master...
I20260807 08:47:58.595280 13554 heartbeater.cc:511] Master 127.4.159.190:38689 requested a full tablet report, sending...
I20260807 08:47:58.595599 12870 ts_manager.cc:194] Registered new tserver with Master: b4b8fd6650c7435ab1c390e4921705cf (127.4.159.131:42325)
I20260807 08:47:58.595981 12870 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.159.131@KRBTEST.COM'} at 127.4.159.131:39273
I20260807 08:47:58.604606 4734 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786092453486433-4734-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786092453486433-4734-0/krb5kdc/test-admin.keytab.
Aug 07 08:47:58 dist-test-slave-6zbq krb5kdc[12801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092478, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:47:58.621568 4734 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260807 08:47:58.622946 13565 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:47:58.622290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57306 (local address 127.4.159.129:39179)
0807 08:47:58.622456 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:58.622461 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:58.622472 (+ 11us) server_negotiation.cc:407] Connection header received
0807 08:47:58.622495 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:58.622497 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:58.622510 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:58.622574 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:47:58.622880 (+ 306us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57306: BlockingRecv error: recv got EOF from 127.0.0.1:57306 (error 108)
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":106,"threads_started":1}
W20260807 08:47:58.623054 4734 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786092453486433-4734-0/krb5kdc/krb5cc' of type FILE
Aug 07 08:47:58 dist-test-slave-6zbq krb5kdc[12801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092478, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:47:58.623934 4734 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 08:47:58 dist-test-slave-6zbq krb5kdc[12801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092478, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:47:58.624706 4734 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 08:47:58 dist-test-slave-6zbq krb5kdc[12801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092478, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.159.129@KRBTEST.COM
I20260807 08:47:58.631457 13565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:58.625141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57310 (local address 127.4.159.129:39179)
0807 08:47:58.625194 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:58.625195 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:58.625203 (+ 8us) server_negotiation.cc:407] Connection header received
0807 08:47:58.625242 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:58.625242 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:58.625250 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:58.625295 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:47:58.625624 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:58.626372 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:58.627045 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:58.627375 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:58.627429 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:58.629276 (+ 1847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:47:58.629285 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:47:58.629285 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:47:58.629291 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:47:58.630584 (+ 1293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:58.630870 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:58.630872 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:58.630872 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:58.630920 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:58.631044 (+ 124us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:58.631045 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:58.631045 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:58.631161 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:47:58.631298 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:58.631331 (+ 33us) server_negotiation.cc:299] Negotiation successful
0807 08:47:58.631388 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
W20260807 08:47:58.633018 13565 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:47:58.632559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57322 (local address 127.4.159.129:39179)
0807 08:47:58.632614 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:58.632616 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:58.632624 (+ 8us) server_negotiation.cc:407] Connection header received
0807 08:47:58.632681 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:58.632682 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:58.632692 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:58.632735 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:47:58.632976 (+ 241us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57322: BlockingRecv error: recv got EOF from 127.0.0.1:57322 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
W20260807 08:47:58.633064 4734 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786092453486433-4734-0/krb5kdc/krb5cc' of type FILE
Aug 07 08:47:58 dist-test-slave-6zbq krb5kdc[12801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092478, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:47:58.633913 4734 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 08:47:58 dist-test-slave-6zbq krb5kdc[12801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092478, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:47:58.634673 4734 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 08:47:58 dist-test-slave-6zbq krb5kdc[12801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092478, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.159.129@KRBTEST.COM
I20260807 08:47:58.640856 13565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:58.635156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57324 (local address 127.4.159.129:39179)
0807 08:47:58.635192 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:58.635194 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:58.635230 (+ 36us) server_negotiation.cc:407] Connection header received
0807 08:47:58.635298 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:58.635299 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:58.635309 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:58.635355 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:47:58.635715 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:58.636186 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:58.636861 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:58.637116 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:58.637177 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:58.639104 (+ 1927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:47:58.639105 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:47:58.639105 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:47:58.639111 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:47:58.640112 (+ 1001us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:58.640385 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:58.640385 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:58.640385 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:58.640427 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:58.640542 (+ 115us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:58.640542 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:58.640543 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:58.640618 (+ 75us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:47:58.640676 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:58.640748 (+ 72us) server_negotiation.cc:299] Negotiation successful
0807 08:47:58.640800 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
W20260807 08:47:58.642180 13565 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:47:58.641687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57336 (local address 127.4.159.129:39179)
0807 08:47:58.641723 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:58.641724 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:58.641781 (+ 57us) server_negotiation.cc:407] Connection header received
0807 08:47:58.641845 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:58.641845 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:58.641854 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:58.641899 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:47:58.642137 (+ 238us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57336: BlockingRecv error: recv got EOF from 127.0.0.1:57336 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260807 08:47:58.642251 4734 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786092453486433-4734-0/krb5kdc/krb5cc' of type FILE
Aug 07 08:47:58 dist-test-slave-6zbq krb5kdc[12801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092478, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:47:58.643095 4734 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 08:47:58 dist-test-slave-6zbq krb5kdc[12801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092478, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:47:58.643827 4734 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 08:47:58 dist-test-slave-6zbq krb5kdc[12801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092478, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.159.129@KRBTEST.COM
I20260807 08:47:58.649360 13565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:47:58.644258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57340 (local address 127.4.159.129:39179)
0807 08:47:58.644313 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0807 08:47:58.644315 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:47:58.644334 (+ 19us) server_negotiation.cc:407] Connection header received
0807 08:47:58.644380 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:47:58.644380 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:47:58.644387 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:47:58.644428 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:47:58.644719 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:58.645142 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:58.645731 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:47:58.645892 (+ 161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:47:58.645933 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:47:58.647615 (+ 1682us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:47:58.647616 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:47:58.647617 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:47:58.647622 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:47:58.648600 (+ 978us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:58.648872 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:58.648873 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:58.648873 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:58.648913 (+ 40us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:47:58.649030 (+ 117us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:47:58.649030 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:47:58.649031 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:47:58.649099 (+ 68us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:47:58.649155 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 08:47:58.649238 (+ 83us) server_negotiation.cc:299] Negotiation successful
0807 08:47:58.649286 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260807 08:47:58.649932 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 12964
I20260807 08:47:58.661617 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 13160
I20260807 08:47:58.673688 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 13363
I20260807 08:47:58.684314 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 12820
2026-08-07T08:47:58Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (1982 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:47:58 dist-test-slave-6zbq krb5kdc[13584](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:47:58 dist-test-slave-6zbq krb5kdc[13584](info): set up 2 sockets
Aug 07 08:47:58 dist-test-slave-6zbq krb5kdc[13584](info): commencing operation
krb5kdc: starting...
W20260807 08:47:59.803721 4734 mini_kdc.cc:121] Time spent starting KDC: real 1.094s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/krb5kdc/test-user.keytab.
Aug 07 08:47:59 dist-test-slave-6zbq krb5kdc[13584](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092479, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:47:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:47:59Z Disabled control of system clock
WARNING: no policy specified for oryx/127.4.159.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.159.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.159.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/krb5kdc/oryx_127.4.159.190.keytab.
Entry for principal oryx/127.4.159.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/krb5kdc/oryx_127.4.159.190.keytab.
I20260807 08:47:59.912716 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.159.190:37483
--webserver_interface=127.4.159.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/krb5kdc/oryx_127.4.159.190.keytab
--principal=oryx/127.4.159.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:45003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.159.190:37483
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:47:59.991819 13603 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:47:59.992008 13603 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:47:59.992038 13603 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:47:59.993124 13603 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:47:59.993161 13603 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:47:59.993180 13603 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:47:59.993196 13603 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:47:59.993211 13603 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:47:59.994665 13603 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:45003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.159.190:37483
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/krb5kdc/oryx_127.4.159.190.keytab
--principal=oryx/127.4.159.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.159.190:37483
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.159.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13603
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:47:59.994896 13603 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:47:59.995091 13603 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:47:59.999473 13614 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:47:59.999501 13617 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:47:59.999567 13603 server_base.cc:1034] running on GCE node
W20260807 08:47:59.999481 13612 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:47:59.999949 13603 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:48:00.000252 13603 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:48:00.001379 13603 hybrid_clock.cc:648] HybridClock initialized: now 1786092480001364 us; error 32 us; skew 500 ppm
Aug 07 08:48:00 dist-test-slave-6zbq krb5kdc[13584](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092480, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.159.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:48:00.008473 13603 init.cc:377] Logged in from keytab as oryx/127.4.159.190@KRBTEST.COM (short username oryx)
I20260807 08:48:00.008931 13603 webserver.cc:460] Webserver started at http://127.4.159.190:35213/ using document root <none> and password file <none>
I20260807 08:48:00.009160 13603 fs_manager.cc:359] Metadata directory not provided
I20260807 08:48:00.009212 13603 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:48:00.009342 13603 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:48:00.010442 13603 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/master-0/data/instance:
uuid: "82a928c4f0234124b0567479c875bb25"
format_stamp: "Formatted at 2026-08-07 08:48:00 on dist-test-slave-6zbq"
server_key: "3d5fbc50a1260f03a8b12adc2fcd36dc"
server_key_iv: "fa821011f77f07079623b78a7a3cc288"
server_key_version: "encryptionkey@0"
I20260807 08:48:00.010783 13603 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/master-0/wal/instance:
uuid: "82a928c4f0234124b0567479c875bb25"
format_stamp: "Formatted at 2026-08-07 08:48:00 on dist-test-slave-6zbq"
server_key: "3d5fbc50a1260f03a8b12adc2fcd36dc"
server_key_iv: "fa821011f77f07079623b78a7a3cc288"
server_key_version: "encryptionkey@0"
I20260807 08:48:00.011987 13603 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:48:00.012631 13626 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:48:00.012778 13603 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 08:48:00.012862 13603 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/master-0/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/master-0/wal
uuid: "82a928c4f0234124b0567479c875bb25"
format_stamp: "Formatted at 2026-08-07 08:48:00 on dist-test-slave-6zbq"
server_key: "3d5fbc50a1260f03a8b12adc2fcd36dc"
server_key_iv: "fa821011f77f07079623b78a7a3cc288"
server_key_version: "encryptionkey@0"
I20260807 08:48:00.012920 13603 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:48:00.028424 13603 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:48:00.030036 13603 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:48:00.030170 13603 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:48:00.033885 13603 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.190:37483
I20260807 08:48:00.033923 13711 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.190:37483 every 8 connection(s)
I20260807 08:48:00.034233 13603 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
I20260807 08:48:00.035202 13713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:48:00.036960 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 13603
I20260807 08:48:00.037076 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/master-0/wal/instance
I20260807 08:48:00.037266 4734 external_mini_cluster.cc:1428] Setting key 1775967a8b0c2529829b00f605e71cf6
I20260807 08:48:00.037559 13713 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82a928c4f0234124b0567479c875bb25: Bootstrap starting.
I20260807 08:48:00.038503 13713 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 82a928c4f0234124b0567479c875bb25: Neither blocks nor log segments found. Creating new log.
I20260807 08:48:00.038839 13713 log.cc:824] T 00000000000000000000000000000000 P 82a928c4f0234124b0567479c875bb25: Log is configured to *not* fsync() on all Append() calls
I20260807 08:48:00.039562 13713 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82a928c4f0234124b0567479c875bb25: No bootstrap required, opened a new log
I20260807 08:48:00.041747 13713 raft_consensus.cc:348] T 00000000000000000000000000000000 P 82a928c4f0234124b0567479c875bb25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82a928c4f0234124b0567479c875bb25" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 37483 } }
I20260807 08:48:00.041910 13713 raft_consensus.cc:374] T 00000000000000000000000000000000 P 82a928c4f0234124b0567479c875bb25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:48:00.041954 13713 raft_consensus.cc:729] T 00000000000000000000000000000000 P 82a928c4f0234124b0567479c875bb25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82a928c4f0234124b0567479c875bb25, State: Initialized, Role: FOLLOWER
I20260807 08:48:00.042078 13713 consensus_queue.cc:260] T 00000000000000000000000000000000 P 82a928c4f0234124b0567479c875bb25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82a928c4f0234124b0567479c875bb25" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 37483 } }
I20260807 08:48:00.042177 13713 raft_consensus.cc:388] T 00000000000000000000000000000000 P 82a928c4f0234124b0567479c875bb25 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:48:00.042219 13713 raft_consensus.cc:482] T 00000000000000000000000000000000 P 82a928c4f0234124b0567479c875bb25 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:48:00.042260 13713 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 82a928c4f0234124b0567479c875bb25 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:48:00.042881 13713 raft_consensus.cc:504] T 00000000000000000000000000000000 P 82a928c4f0234124b0567479c875bb25 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82a928c4f0234124b0567479c875bb25" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 37483 } }
I20260807 08:48:00.042970 13713 leader_election.cc:302] T 00000000000000000000000000000000 P 82a928c4f0234124b0567479c875bb25 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 82a928c4f0234124b0567479c875bb25; no voters:
I20260807 08:48:00.043121 13713 leader_election.cc:288] T 00000000000000000000000000000000 P 82a928c4f0234124b0567479c875bb25 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:48:00.043190 13724 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 82a928c4f0234124b0567479c875bb25 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:48:00.043342 13724 raft_consensus.cc:686] T 00000000000000000000000000000000 P 82a928c4f0234124b0567479c875bb25 [term 1 LEADER]: Becoming Leader. State: Replica: 82a928c4f0234124b0567479c875bb25, State: Running, Role: LEADER
I20260807 08:48:00.043361 13713 sys_catalog.cc:564] T 00000000000000000000000000000000 P 82a928c4f0234124b0567479c875bb25 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:48:00.043438 13724 consensus_queue.cc:237] T 00000000000000000000000000000000 P 82a928c4f0234124b0567479c875bb25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82a928c4f0234124b0567479c875bb25" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 37483 } }
I20260807 08:48:00.043778 13724 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82a928c4f0234124b0567479c875bb25 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 82a928c4f0234124b0567479c875bb25. Latest consensus state: current_term: 1 leader_uuid: "82a928c4f0234124b0567479c875bb25" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82a928c4f0234124b0567479c875bb25" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 37483 } } }
I20260807 08:48:00.043859 13724 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82a928c4f0234124b0567479c875bb25 [sys.catalog]: This master's current role is: LEADER
I20260807 08:48:00.043795 13726 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82a928c4f0234124b0567479c875bb25 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "82a928c4f0234124b0567479c875bb25" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82a928c4f0234124b0567479c875bb25" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 37483 } } }
I20260807 08:48:00.044024 13726 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82a928c4f0234124b0567479c875bb25 [sys.catalog]: This master's current role is: LEADER
Aug 07 08:48:00 dist-test-slave-6zbq krb5kdc[13584](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092479, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM
I20260807 08:48:00.044261 13732 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:48:00.044752 13732 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:48:00.047012 13732 catalog_manager.cc:1347] Generated new cluster ID: 6e5ed93ced5c45e9b23a567a01055589
I20260807 08:48:00.047077 13732 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:48:00.048312 13722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:00.037766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44824 (local address 127.4.159.190:37483)
0807 08:48:00.037928 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:00.037937 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:00.037968 (+ 31us) server_negotiation.cc:407] Connection header received
0807 08:48:00.038586 (+ 618us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:00.038595 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:00.038665 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:00.038821 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:00.039383 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:00.040735 (+ 1352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:00.041444 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:00.042030 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:00.042155 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:00.044678 (+ 2523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:48:00.044683 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:48:00.044684 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:48:00.044697 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:48:00.046764 (+ 2067us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:48:00.047183 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:48:00.047187 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:48:00.047187 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:48:00.047279 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:48:00.047457 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:48:00.047458 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:48:00.047458 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:48:00.047701 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:48:00.047838 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0807 08:48:00.047961 (+ 123us) server_negotiation.cc:299] Negotiation successful
0807 08:48:00.048099 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":66,"threads_started":1}
I20260807 08:48:00.057952 13732 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:48:00.058660 13732 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:48:00.066578 13732 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 82a928c4f0234124b0567479c875bb25: Generated new TSK 0
I20260807 08:48:00.066736 13732 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.4.159.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.159.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.159.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/krb5kdc/oryx_127.4.159.129.keytab.
Entry for principal oryx/127.4.159.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/krb5kdc/oryx_127.4.159.129.keytab.
I20260807 08:48:00.091204 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.159.129:0
--local_ip_for_outbound_sockets=127.4.159.129
--webserver_interface=127.4.159.129
--webserver_port=0
--tserver_master_addrs=127.4.159.190:37483
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/krb5kdc/oryx_127.4.159.129.keytab
--principal=oryx/127.4.159.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:45003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:48:00.177310 13756 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:48:00.177512 13756 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:48:00.177534 13756 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:48:00.178685 13756 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:48:00.178726 13756 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:48:00.178753 13756 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.159.129
I20260807 08:48:00.180182 13756 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:45003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/krb5kdc/oryx_127.4.159.129.keytab
--principal=oryx/127.4.159.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.159.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.159.129
--webserver_port=0
--tserver_master_addrs=127.4.159.190:37483
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13756
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.159.129
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:48:00.180403 13756 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:48:00.180615 13756 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:48:00.185266 13762 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:48:00.185273 13764 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:48:00.185461 13756 server_base.cc:1034] running on GCE node
W20260807 08:48:00.185469 13761 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:48:00.185688 13756 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:48:00.185914 13756 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:48:00.187047 13756 hybrid_clock.cc:648] HybridClock initialized: now 1786092480187033 us; error 28 us; skew 500 ppm
Aug 07 08:48:00 dist-test-slave-6zbq krb5kdc[13584](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092480, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.159.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:48:00.189323 13756 init.cc:377] Logged in from keytab as oryx/127.4.159.129@KRBTEST.COM (short username oryx)
I20260807 08:48:00.189721 13756 webserver.cc:460] Webserver started at http://127.4.159.129:43369/ using document root <none> and password file <none>
I20260807 08:48:00.189906 13756 fs_manager.cc:359] Metadata directory not provided
I20260807 08:48:00.189952 13756 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:48:00.190074 13756 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:48:00.191123 13756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/ts-0/data/instance:
uuid: "67262c924f4e4493bd8dcab7c92e6aaa"
format_stamp: "Formatted at 2026-08-07 08:48:00 on dist-test-slave-6zbq"
server_key: "efa78947e318008363fee6339f6e564f"
server_key_iv: "7dfd3ed3e1133659e024142d8ff240c2"
server_key_version: "encryptionkey@0"
I20260807 08:48:00.191447 13756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/ts-0/wal/instance:
uuid: "67262c924f4e4493bd8dcab7c92e6aaa"
format_stamp: "Formatted at 2026-08-07 08:48:00 on dist-test-slave-6zbq"
server_key: "efa78947e318008363fee6339f6e564f"
server_key_iv: "7dfd3ed3e1133659e024142d8ff240c2"
server_key_version: "encryptionkey@0"
I20260807 08:48:00.192643 13756 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260807 08:48:00.193252 13772 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:48:00.193405 13756 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 08:48:00.193488 13756 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/ts-0/wal
uuid: "67262c924f4e4493bd8dcab7c92e6aaa"
format_stamp: "Formatted at 2026-08-07 08:48:00 on dist-test-slave-6zbq"
server_key: "efa78947e318008363fee6339f6e564f"
server_key_iv: "7dfd3ed3e1133659e024142d8ff240c2"
server_key_version: "encryptionkey@0"
I20260807 08:48:00.193547 13756 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:48:00.211140 13756 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:48:00.213270 13756 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:48:00.213409 13756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:48:00.213622 13756 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:48:00.213925 13756 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:48:00.213959 13756 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:48:00.213989 13756 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:48:00.214012 13756 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:48:00.219763 13756 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.129:35375
I20260807 08:48:00.219805 13947 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.129:35375 every 8 connection(s)
I20260807 08:48:00.220124 13756 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/ts-0/data/info.pb
I20260807 08:48:00.225741 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 13756
Aug 07 08:48:00 dist-test-slave-6zbq krb5kdc[13584](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092480, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.159.129@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM
I20260807 08:48:00.225832 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/ts-0/wal/instance
I20260807 08:48:00.226017 4734 external_mini_cluster.cc:1428] Setting key c58da36dc9322aa949d4cc19b5447c65
I20260807 08:48:00.228780 13722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:00.220783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.129:35981 (local address 127.4.159.190:37483)
0807 08:48:00.220839 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:00.220841 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:00.221039 (+ 198us) server_negotiation.cc:407] Connection header received
0807 08:48:00.221561 (+ 522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:00.221562 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:00.221575 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:00.221636 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:00.222586 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:00.223096 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:00.224041 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:00.224253 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:00.224309 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:00.226247 (+ 1938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:48:00.226248 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:48:00.226249 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:48:00.226255 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:48:00.227559 (+ 1304us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:48:00.227925 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:48:00.227926 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:48:00.227926 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:48:00.227981 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:48:00.228194 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:48:00.228195 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:48:00.228195 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:48:00.228302 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:48:00.228357 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 08:48:00.228604 (+ 247us) server_negotiation.cc:299] Negotiation successful
0807 08:48:00.228687 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260807 08:48:00.228955 13950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:00.220758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:37483 (local address 127.4.159.129:35981)
0807 08:48:00.220923 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:00.220959 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:00.221464 (+ 505us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:00.221713 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:00.221716 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:00.222266 (+ 550us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:48:00.222494 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:00.222500 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:00.223180 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:00.223181 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:00.223954 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:00.223956 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:00.224323 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:00.224324 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:00.224435 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:00.224840 (+ 405us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:48:00.224848 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:48:00.226156 (+ 1308us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:48:00.227638 (+ 1482us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:48:00.227646 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:48:00.227648 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:48:00.227860 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:48:00.228032 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:48:00.228033 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:48:00.228034 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:48:00.228140 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:48:00.228355 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:48:00.228362 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:48:00.228491 (+ 129us) client_negotiation.cc:769] Sending connection context
0807 08:48:00.228577 (+ 86us) client_negotiation.cc:240] Negotiation successful
0807 08:48:00.228736 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":85,"threads_started":1}
I20260807 08:48:00.229600 13948 heartbeater.cc:347] Connected to a master server at 127.4.159.190:37483
I20260807 08:48:00.229682 13948 heartbeater.cc:464] Registering TS with master...
I20260807 08:48:00.229859 13948 heartbeater.cc:511] Master 127.4.159.190:37483 requested a full tablet report, sending...
I20260807 08:48:00.230391 13654 ts_manager.cc:194] Registered new tserver with Master: 67262c924f4e4493bd8dcab7c92e6aaa (127.4.159.129:35375)
I20260807 08:48:00.230995 13654 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.4.159.129@KRBTEST.COM'} at 127.4.159.129:35981
WARNING: no policy specified for oryx/127.4.159.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.159.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.159.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/krb5kdc/oryx_127.4.159.130.keytab.
Entry for principal oryx/127.4.159.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/krb5kdc/oryx_127.4.159.130.keytab.
W20260807 08:48:00.937309 4734 mini_kdc.cc:262] Time spent creating service keytab for oryx/127.4.159.130: real 0.710s user 0.001s sys 0.002s
I20260807 08:48:00.937680 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.159.130:0
--local_ip_for_outbound_sockets=127.4.159.130
--webserver_interface=127.4.159.130
--webserver_port=0
--tserver_master_addrs=127.4.159.190:37483
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/krb5kdc/oryx_127.4.159.130.keytab
--principal=oryx/127.4.159.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:45003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:48:01.019693 13954 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:48:01.019894 13954 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:48:01.019925 13954 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:48:01.021026 13954 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:48:01.021067 13954 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:48:01.021118 13954 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.159.130
I20260807 08:48:01.022554 13954 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:45003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/krb5kdc/oryx_127.4.159.130.keytab
--principal=oryx/127.4.159.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.159.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.159.130
--webserver_port=0
--tserver_master_addrs=127.4.159.190:37483
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13954
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.159.130
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:48:01.022784 13954 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:48:01.022989 13954 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:48:01.027662 13963 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:48:01.027791 13954 server_base.cc:1034] running on GCE node
W20260807 08:48:01.027720 13966 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:48:01.027662 13962 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:48:01.028069 13954 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:48:01.028420 13954 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:48:01.029572 13954 hybrid_clock.cc:648] HybridClock initialized: now 1786092481029557 us; error 34 us; skew 500 ppm
Aug 07 08:48:01 dist-test-slave-6zbq krb5kdc[13584](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092481, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.159.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:48:01.032589 13954 init.cc:377] Logged in from keytab as oryx/127.4.159.130@KRBTEST.COM (short username oryx)
I20260807 08:48:01.032979 13954 webserver.cc:460] Webserver started at http://127.4.159.130:35819/ using document root <none> and password file <none>
I20260807 08:48:01.033186 13954 fs_manager.cc:359] Metadata directory not provided
I20260807 08:48:01.033233 13954 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:48:01.033356 13954 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:48:01.034322 13954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/ts-1/data/instance:
uuid: "bed5f8e9f0ba46ea97101418a4f0affe"
format_stamp: "Formatted at 2026-08-07 08:48:01 on dist-test-slave-6zbq"
server_key: "3e9967887dc223e89823909cb109304b"
server_key_iv: "e65e9138feb753c604ed8b3c120937ba"
server_key_version: "encryptionkey@0"
I20260807 08:48:01.034687 13954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/ts-1/wal/instance:
uuid: "bed5f8e9f0ba46ea97101418a4f0affe"
format_stamp: "Formatted at 2026-08-07 08:48:01 on dist-test-slave-6zbq"
server_key: "3e9967887dc223e89823909cb109304b"
server_key_iv: "e65e9138feb753c604ed8b3c120937ba"
server_key_version: "encryptionkey@0"
I20260807 08:48:01.035996 13954 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:48:01.036726 13973 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:48:01.036870 13954 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 08:48:01.036955 13954 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/ts-1/wal
uuid: "bed5f8e9f0ba46ea97101418a4f0affe"
format_stamp: "Formatted at 2026-08-07 08:48:01 on dist-test-slave-6zbq"
server_key: "3e9967887dc223e89823909cb109304b"
server_key_iv: "e65e9138feb753c604ed8b3c120937ba"
server_key_version: "encryptionkey@0"
I20260807 08:48:01.037017 13954 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:48:01.062599 13954 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:48:01.064590 13954 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:48:01.064697 13954 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:48:01.064905 13954 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:48:01.065181 13954 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:48:01.065217 13954 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:48:01.065244 13954 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:48:01.065275 13954 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:48:01.070884 13954 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.130:39079
I20260807 08:48:01.071038 14168 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.130:39079 every 8 connection(s)
I20260807 08:48:01.071234 13954 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/ts-1/data/info.pb
I20260807 08:48:01.073094 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 13954
I20260807 08:48:01.073180 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/ts-1/wal/instance
I20260807 08:48:01.073383 4734 external_mini_cluster.cc:1428] Setting key 14b34da257e809c2b209bab69b231a61
Aug 07 08:48:01 dist-test-slave-6zbq krb5kdc[13584](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092481, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.159.130@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM
I20260807 08:48:01.082090 14173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:01.071949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.130:57603 (local address 127.4.159.190:37483)
0807 08:48:01.072100 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:01.072103 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:01.072259 (+ 156us) server_negotiation.cc:407] Connection header received
0807 08:48:01.072973 (+ 714us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:01.072974 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:01.072989 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:01.073045 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:01.074285 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:01.075005 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:01.076349 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:01.076555 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:01.076621 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:01.079270 (+ 2649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:48:01.079272 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:48:01.079272 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:48:01.079279 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:48:01.080659 (+ 1380us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:48:01.081137 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:48:01.081137 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:48:01.081138 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:48:01.081196 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:48:01.081446 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:48:01.081446 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:48:01.081446 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:48:01.081545 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:48:01.081590 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0807 08:48:01.081935 (+ 345us) server_negotiation.cc:299] Negotiation successful
0807 08:48:01.082012 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":70,"threads_started":1}
I20260807 08:48:01.082298 14171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:01.071876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:37483 (local address 127.4.159.130:57603)
0807 08:48:01.072111 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:01.072166 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:01.072704 (+ 538us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:01.073115 (+ 411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:01.073119 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:01.073840 (+ 721us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:48:01.074142 (+ 302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:01.074151 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:01.075246 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:01.075247 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:01.076203 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:01.076207 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:01.076647 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:01.076648 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:01.076792 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:01.077385 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:48:01.077395 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:48:01.079153 (+ 1758us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:48:01.080741 (+ 1588us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:48:01.080749 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:48:01.080752 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:48:01.081043 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:48:01.081250 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:48:01.081251 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:48:01.081252 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:48:01.081389 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:48:01.081599 (+ 210us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:48:01.081605 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:48:01.081768 (+ 163us) client_negotiation.cc:769] Sending connection context
0807 08:48:01.081889 (+ 121us) client_negotiation.cc:240] Negotiation successful
0807 08:48:01.082057 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":120,"threads_started":1}
I20260807 08:48:01.082845 14169 heartbeater.cc:347] Connected to a master server at 127.4.159.190:37483
I20260807 08:48:01.082939 14169 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for oryx/127.4.159.131@KRBTEST.COM; defaulting to no policy
I20260807 08:48:01.083120 14169 heartbeater.cc:511] Master 127.4.159.190:37483 requested a full tablet report, sending...
I20260807 08:48:01.083496 13654 ts_manager.cc:194] Registered new tserver with Master: bed5f8e9f0ba46ea97101418a4f0affe (127.4.159.130:39079)
I20260807 08:48:01.083832 13654 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.4.159.130@KRBTEST.COM'} at 127.4.159.130:57603
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.159.131@KRBTEST.COM" created.
I20260807 08:48:01.231851 13948 heartbeater.cc:503] Master 127.4.159.190:37483 was elected leader, sending a full tablet report...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.159.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/krb5kdc/oryx_127.4.159.131.keytab.
Entry for principal oryx/127.4.159.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/krb5kdc/oryx_127.4.159.131.keytab.
W20260807 08:48:01.974539 4734 mini_kdc.cc:262] Time spent creating service keytab for oryx/127.4.159.131: real 0.900s user 0.002s sys 0.001s
I20260807 08:48:01.974888 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.159.131:0
--local_ip_for_outbound_sockets=127.4.159.131
--webserver_interface=127.4.159.131
--webserver_port=0
--tserver_master_addrs=127.4.159.190:37483
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/krb5kdc/oryx_127.4.159.131.keytab
--principal=oryx/127.4.159.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:45003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:48:02.054708 14177 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:48:02.054884 14177 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:48:02.054912 14177 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:48:02.056006 14177 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:48:02.056046 14177 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:48:02.056079 14177 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.159.131
I20260807 08:48:02.057478 14177 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:45003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/krb5kdc/oryx_127.4.159.131.keytab
--principal=oryx/127.4.159.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.159.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.159.131
--webserver_port=0
--tserver_master_addrs=127.4.159.190:37483
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14177
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.159.131
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:48:02.057709 14177 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:48:02.057904 14177 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:48:02.062474 14189 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:48:02.062575 14177 server_base.cc:1034] running on GCE node
W20260807 08:48:02.062503 14184 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:48:02.062475 14186 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:48:02.062909 14177 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:48:02.063139 14177 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:48:02.064270 14177 hybrid_clock.cc:648] HybridClock initialized: now 1786092482064257 us; error 22 us; skew 500 ppm
Aug 07 08:48:02 dist-test-slave-6zbq krb5kdc[13584](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092482, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.159.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:48:02.066568 14177 init.cc:377] Logged in from keytab as oryx/127.4.159.131@KRBTEST.COM (short username oryx)
I20260807 08:48:02.067011 14177 webserver.cc:460] Webserver started at http://127.4.159.131:40137/ using document root <none> and password file <none>
I20260807 08:48:02.067234 14177 fs_manager.cc:359] Metadata directory not provided
I20260807 08:48:02.067281 14177 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:48:02.067402 14177 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:48:02.068356 14177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/ts-2/data/instance:
uuid: "48c69edfbb324b37ace3f551898f2220"
format_stamp: "Formatted at 2026-08-07 08:48:02 on dist-test-slave-6zbq"
server_key: "c17f15525e12913055c2425a4b38b0a5"
server_key_iv: "e0a0cf1c150d54d28a6fab46d4249a36"
server_key_version: "encryptionkey@0"
I20260807 08:48:02.068663 14177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/ts-2/wal/instance:
uuid: "48c69edfbb324b37ace3f551898f2220"
format_stamp: "Formatted at 2026-08-07 08:48:02 on dist-test-slave-6zbq"
server_key: "c17f15525e12913055c2425a4b38b0a5"
server_key_iv: "e0a0cf1c150d54d28a6fab46d4249a36"
server_key_version: "encryptionkey@0"
I20260807 08:48:02.069944 14177 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 08:48:02.070643 14201 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:48:02.070806 14177 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 08:48:02.070894 14177 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/ts-2/wal
uuid: "48c69edfbb324b37ace3f551898f2220"
format_stamp: "Formatted at 2026-08-07 08:48:02 on dist-test-slave-6zbq"
server_key: "c17f15525e12913055c2425a4b38b0a5"
server_key_iv: "e0a0cf1c150d54d28a6fab46d4249a36"
server_key_version: "encryptionkey@0"
I20260807 08:48:02.070963 14177 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:48:02.084594 14169 heartbeater.cc:503] Master 127.4.159.190:37483 was elected leader, sending a full tablet report...
I20260807 08:48:02.091475 14177 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:48:02.092999 14177 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:48:02.093122 14177 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:48:02.093307 14177 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:48:02.093571 14177 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:48:02.093604 14177 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:48:02.093631 14177 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:48:02.093655 14177 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:48:02.099121 14177 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.131:43209
I20260807 08:48:02.099180 14367 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.131:43209 every 8 connection(s)
I20260807 08:48:02.099438 14177 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/ts-2/data/info.pb
I20260807 08:48:02.099773 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 14177
I20260807 08:48:02.099876 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/ts-2/wal/instance
I20260807 08:48:02.100087 4734 external_mini_cluster.cc:1428] Setting key eb553f787438bb1a7fe8687061129a8f
Aug 07 08:48:02 dist-test-slave-6zbq krb5kdc[13584](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092482, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.159.131@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM
I20260807 08:48:02.107825 14371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:02.100098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.131:32793 (local address 127.4.159.190:37483)
0807 08:48:02.100332 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:02.100334 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:02.100365 (+ 31us) server_negotiation.cc:407] Connection header received
0807 08:48:02.100878 (+ 513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:02.100879 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:02.100892 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:02.100952 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:02.101810 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:02.102309 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:02.103236 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:02.103428 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:02.103477 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:02.105371 (+ 1894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:48:02.105373 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:48:02.105373 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:48:02.105381 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:48:02.106602 (+ 1221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:48:02.106957 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:48:02.106958 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:48:02.106958 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:48:02.107015 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:48:02.107223 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:48:02.107224 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:48:02.107225 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:48:02.107334 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:48:02.107399 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0807 08:48:02.107639 (+ 240us) server_negotiation.cc:299] Negotiation successful
0807 08:48:02.107727 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":213,"thread_start_us":65,"threads_started":1}
I20260807 08:48:02.107964 14372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:02.100085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:37483 (local address 127.4.159.131:32793)
0807 08:48:02.100250 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:02.100298 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:02.100805 (+ 507us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:02.101017 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:02.101021 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:02.101557 (+ 536us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:48:02.101740 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:02.101745 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:02.102399 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:02.102400 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:02.103169 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:02.103172 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:02.103485 (+ 313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:02.103486 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:02.103583 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:02.103982 (+ 399us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:48:02.103989 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:48:02.105278 (+ 1289us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:48:02.106675 (+ 1397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:48:02.106682 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:48:02.106684 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:48:02.106890 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:48:02.107068 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:48:02.107069 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:48:02.107069 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:48:02.107168 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:48:02.107389 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:48:02.107396 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:48:02.107522 (+ 126us) client_negotiation.cc:769] Sending connection context
0807 08:48:02.107611 (+ 89us) client_negotiation.cc:240] Negotiation successful
0807 08:48:02.107767 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":90,"threads_started":1}
I20260807 08:48:02.108350 14368 heartbeater.cc:347] Connected to a master server at 127.4.159.190:37483
I20260807 08:48:02.108431 14368 heartbeater.cc:464] Registering TS with master...
I20260807 08:48:02.108585 14368 heartbeater.cc:511] Master 127.4.159.190:37483 requested a full tablet report, sending...
I20260807 08:48:02.108873 13654 ts_manager.cc:194] Registered new tserver with Master: 48c69edfbb324b37ace3f551898f2220 (127.4.159.131:43209)
I20260807 08:48:02.109200 13654 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.4.159.131@KRBTEST.COM'} at 127.4.159.131:32793
I20260807 08:48:02.111720 4734 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.4.159.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.159.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.159.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/krb5kdc/oryx_127.4.159.189.keytab.
Entry for principal oryx/127.4.159.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/krb5kdc/oryx_127.4.159.189.keytab.
I20260807 08:48:02.136466 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.159.189:37399
--webserver_interface=127.4.159.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/krb5kdc/oryx_127.4.159.189.keytab
--principal=oryx/127.4.159.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:45003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.159.190:37483,127.4.159.189:37399
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:48:02.215123 14377 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:48:02.215310 14377 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:48:02.215341 14377 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:48:02.216441 14377 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:48:02.216483 14377 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:48:02.216502 14377 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:48:02.216518 14377 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:48:02.216534 14377 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:48:02.217962 14377 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:45003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.4.159.190:37483,127.4.159.189:37399
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/krb5kdc/oryx_127.4.159.189.keytab
--principal=oryx/127.4.159.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.159.189:37399
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.4.159.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14377
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
W20260807 08:48:02.218066 14377 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.4.159.190:37483,127.4.159.189:37399'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260807 08:48:02.218216 14377 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:48:02.218418 14377 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:48:02.222949 14393 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:48:02.222995 14391 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:48:02.223029 14377 server_base.cc:1034] running on GCE node
W20260807 08:48:02.222975 14397 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:48:02.223397 14377 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:48:02.223671 14377 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:48:02.224795 14377 hybrid_clock.cc:648] HybridClock initialized: now 1786092482224783 us; error 26 us; skew 500 ppm
Aug 07 08:48:02 dist-test-slave-6zbq krb5kdc[13584](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092482, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.159.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:48:02.227075 14377 init.cc:377] Logged in from keytab as oryx/127.4.159.189@KRBTEST.COM (short username oryx)
I20260807 08:48:02.227469 14377 webserver.cc:460] Webserver started at http://127.4.159.189:36189/ using document root <none> and password file <none>
I20260807 08:48:02.227658 14377 fs_manager.cc:359] Metadata directory not provided
I20260807 08:48:02.227702 14377 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:48:02.227823 14377 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:48:02.228839 14377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/master-1/data/instance:
uuid: "aae8b08fd1ac4d2699d15a396e7ac8f2"
format_stamp: "Formatted at 2026-08-07 08:48:02 on dist-test-slave-6zbq"
server_key: "61793b2749c53d48bede6856776c4fc9"
server_key_iv: "caa4ba3f31e00156ac36bd53b861e153"
server_key_version: "encryptionkey@0"
I20260807 08:48:02.229153 14377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/master-1/wal/instance:
uuid: "aae8b08fd1ac4d2699d15a396e7ac8f2"
format_stamp: "Formatted at 2026-08-07 08:48:02 on dist-test-slave-6zbq"
server_key: "61793b2749c53d48bede6856776c4fc9"
server_key_iv: "caa4ba3f31e00156ac36bd53b861e153"
server_key_version: "encryptionkey@0"
I20260807 08:48:02.230414 14377 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 08:48:02.231072 14404 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:48:02.231217 14377 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 08:48:02.231297 14377 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/master-1/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/master-1/wal
uuid: "aae8b08fd1ac4d2699d15a396e7ac8f2"
format_stamp: "Formatted at 2026-08-07 08:48:02 on dist-test-slave-6zbq"
server_key: "61793b2749c53d48bede6856776c4fc9"
server_key_iv: "caa4ba3f31e00156ac36bd53b861e153"
server_key_version: "encryptionkey@0"
I20260807 08:48:02.231359 14377 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:48:02.251679 14377 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:48:02.253518 14377 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:48:02.253645 14377 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:48:02.257155 14377 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.189:37399
I20260807 08:48:02.257186 14486 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.189:37399 every 8 connection(s)
I20260807 08:48:02.257462 14377 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/master-1/data/info.pb
I20260807 08:48:02.258361 14487 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:48:02.259747 14487 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.159.190" port: 37483 } has no permanent_uuid. Determining permanent_uuid...
I20260807 08:48:02.260720 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 14377
Aug 07 08:48:02 dist-test-slave-6zbq krb5kdc[13584](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092482, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.159.189@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM
Aug 07 08:48:02 dist-test-slave-6zbq krb5kdc[13584](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092479, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.4.159.189@KRBTEST.COM
I20260807 08:48:02.268740 14371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:02.260343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44838 (local address 127.4.159.190:37483)
0807 08:48:02.260402 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:02.260404 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:02.260580 (+ 176us) server_negotiation.cc:407] Connection header received
0807 08:48:02.261021 (+ 441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:02.261022 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:02.261032 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:02.261096 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:02.261722 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:02.262474 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:02.263691 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:02.264000 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:02.264061 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:02.266037 (+ 1976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:48:02.266038 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:48:02.266039 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:48:02.266046 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:48:02.267467 (+ 1421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:48:02.267819 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:48:02.267820 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:48:02.267820 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:48:02.267880 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:48:02.268074 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:48:02.268074 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:48:02.268075 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:48:02.268235 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:48:02.268286 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0807 08:48:02.268563 (+ 277us) server_negotiation.cc:299] Negotiation successful
0807 08:48:02.268648 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260807 08:48:02.268824 14490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:02.260316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:37483 (local address 127.0.0.1:44838)
0807 08:48:02.260459 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:02.260497 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:02.260927 (+ 430us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:02.261224 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:02.261228 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:02.261465 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:48:02.261636 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:02.261640 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:02.262558 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:02.262559 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:02.263623 (+ 1064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:02.263625 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:02.264076 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:02.264077 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:02.264221 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:02.264641 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:48:02.264649 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:48:02.265954 (+ 1305us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:48:02.267537 (+ 1583us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:48:02.267543 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:48:02.267545 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:48:02.267750 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:48:02.267937 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:48:02.267938 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:48:02.267938 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:48:02.268022 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:48:02.268296 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:48:02.268302 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:48:02.268429 (+ 127us) client_negotiation.cc:769] Sending connection context
0807 08:48:02.268537 (+ 108us) client_negotiation.cc:240] Negotiation successful
0807 08:48:02.268669 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"mutex_wait_us":180,"thread_start_us":67,"threads_started":1}
I20260807 08:48:02.269125 14492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:02.261173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49168 (local address 127.4.159.189:37399)
0807 08:48:02.261327 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:02.261331 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:02.261395 (+ 64us) server_negotiation.cc:407] Connection header received
0807 08:48:02.261447 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:02.261450 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:02.261474 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:02.261537 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:02.262206 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:02.263241 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:02.264059 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:02.264452 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:02.264501 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:02.266628 (+ 2127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:48:02.266635 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:48:02.266635 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:48:02.266641 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:48:02.268043 (+ 1402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:48:02.268354 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:48:02.268357 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:48:02.268357 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:48:02.268404 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:48:02.268570 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:48:02.268571 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:48:02.268571 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:48:02.268732 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:48:02.268893 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0807 08:48:02.268966 (+ 73us) server_negotiation.cc:299] Negotiation successful
0807 08:48:02.269033 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":76,"threads_started":1}
I20260807 08:48:02.269733 14487 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.159.189" port: 37399 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 08:48:02 dist-test-slave-6zbq krb5kdc[13584](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092482, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.159.189@KRBTEST.COM for oryx/127.4.159.189@KRBTEST.COM
I20260807 08:48:02.274838 14490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:02.269908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.189:37399 (local address 127.0.0.1:49182)
0807 08:48:02.269937 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:02.269947 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:02.269986 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:02.270068 (+ 82us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:02.270069 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:02.270174 (+ 105us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:48:02.270200 (+ 26us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:02.270201 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:02.270784 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:02.270785 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:02.271236 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:02.271238 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:02.271506 (+ 268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:02.271507 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:02.271543 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:02.271808 (+ 265us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:48:02.271811 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:48:02.272773 (+ 962us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:48:02.274072 (+ 1299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:48:02.274073 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:48:02.274073 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:48:02.274220 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:48:02.274382 (+ 162us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:48:02.274383 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:48:02.274383 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:48:02.274428 (+ 45us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:48:02.274616 (+ 188us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:48:02.274616 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:48:02.274650 (+ 34us) client_negotiation.cc:769] Sending connection context
0807 08:48:02.274698 (+ 48us) client_negotiation.cc:240] Negotiation successful
0807 08:48:02.274765 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14}
I20260807 08:48:02.274896 14492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:02.269961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49182 (local address 127.4.159.189:37399)
0807 08:48:02.269992 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:02.269994 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:02.270002 (+ 8us) server_negotiation.cc:407] Connection header received
0807 08:48:02.270021 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:02.270021 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:02.270028 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:02.270064 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:02.270246 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:02.270716 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:02.271288 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:02.271460 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:02.271499 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:02.272837 (+ 1338us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:48:02.272838 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:48:02.272838 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:48:02.272843 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:48:02.274016 (+ 1173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:48:02.274271 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:48:02.274271 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:48:02.274272 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:48:02.274313 (+ 41us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:48:02.274478 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:48:02.274478 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:48:02.274479 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:48:02.274568 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:48:02.274621 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0807 08:48:02.274714 (+ 93us) server_negotiation.cc:299] Negotiation successful
0807 08:48:02.274805 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":39,"server-negotiator.queue_time_us":15}
I20260807 08:48:02.275889 14487 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aae8b08fd1ac4d2699d15a396e7ac8f2: Bootstrap starting.
I20260807 08:48:02.276621 14487 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aae8b08fd1ac4d2699d15a396e7ac8f2: Neither blocks nor log segments found. Creating new log.
I20260807 08:48:02.276858 14487 log.cc:824] T 00000000000000000000000000000000 P aae8b08fd1ac4d2699d15a396e7ac8f2: Log is configured to *not* fsync() on all Append() calls
I20260807 08:48:02.277451 14487 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aae8b08fd1ac4d2699d15a396e7ac8f2: No bootstrap required, opened a new log
I20260807 08:48:02.278787 14487 raft_consensus.cc:348] T 00000000000000000000000000000000 P aae8b08fd1ac4d2699d15a396e7ac8f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82a928c4f0234124b0567479c875bb25" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 37483 } } peers { permanent_uuid: "aae8b08fd1ac4d2699d15a396e7ac8f2" member_type: VOTER last_known_addr { host: "127.4.159.189" port: 37399 } }
I20260807 08:48:02.278896 14487 raft_consensus.cc:374] T 00000000000000000000000000000000 P aae8b08fd1ac4d2699d15a396e7ac8f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:48:02.278926 14487 raft_consensus.cc:729] T 00000000000000000000000000000000 P aae8b08fd1ac4d2699d15a396e7ac8f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aae8b08fd1ac4d2699d15a396e7ac8f2, State: Initialized, Role: FOLLOWER
I20260807 08:48:02.279038 14487 consensus_queue.cc:260] T 00000000000000000000000000000000 P aae8b08fd1ac4d2699d15a396e7ac8f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82a928c4f0234124b0567479c875bb25" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 37483 } } peers { permanent_uuid: "aae8b08fd1ac4d2699d15a396e7ac8f2" member_type: VOTER last_known_addr { host: "127.4.159.189" port: 37399 } }
I20260807 08:48:02.279289 14487 sys_catalog.cc:564] T 00000000000000000000000000000000 P aae8b08fd1ac4d2699d15a396e7ac8f2 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:48:02.279259 14495 sys_catalog.cc:455] T 00000000000000000000000000000000 P aae8b08fd1ac4d2699d15a396e7ac8f2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82a928c4f0234124b0567479c875bb25" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 37483 } } peers { permanent_uuid: "aae8b08fd1ac4d2699d15a396e7ac8f2" member_type: VOTER last_known_addr { host: "127.4.159.189" port: 37399 } } }
I20260807 08:48:02.279333 14495 sys_catalog.cc:458] T 00000000000000000000000000000000 P aae8b08fd1ac4d2699d15a396e7ac8f2 [sys.catalog]: This master's current role is: FOLLOWER
W20260807 08:48:02.280732 14516 catalog_manager.cc:1558] T 00000000000000000000000000000000 P aae8b08fd1ac4d2699d15a396e7ac8f2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 08:48:02.280784 14516 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 08:48:02.282248 14377 master_runner.cc:428] Detected that this master aae8b08fd1ac4d2699d15a396e7ac8f2 is joining an existing cluster
I20260807 08:48:02.282317 14377 master_runner.cc:432] Initiating AddMaster RPC to add 127.4.159.189:37399
I20260807 08:48:02.289103 14492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:02.283363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49192 (local address 127.4.159.189:37399)
0807 08:48:02.283394 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:02.283396 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:02.283492 (+ 96us) server_negotiation.cc:407] Connection header received
0807 08:48:02.283566 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:02.283566 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:02.283574 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:02.283597 (+ 23us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:02.283946 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:02.284511 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:02.285527 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:02.285788 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:02.285838 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:02.286753 (+ 915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:48:02.286754 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:48:02.286754 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:48:02.286760 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:48:02.288091 (+ 1331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:48:02.288371 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:48:02.288372 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:48:02.288372 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:48:02.288419 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:48:02.288577 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:48:02.288578 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:48:02.288578 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:48:02.288682 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:48:02.288736 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 08:48:02.288871 (+ 135us) server_negotiation.cc:299] Negotiation successful
0807 08:48:02.288999 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":56,"server-negotiator.queue_time_us":15}
I20260807 08:48:02.289436 14525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:02.283189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.189:37399 (local address 127.0.0.1:49192)
0807 08:48:02.283437 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:02.283446 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:02.283511 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:02.283655 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:02.283655 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:02.283805 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:48:02.283887 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:02.283888 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:02.284583 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:02.284584 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:02.285431 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:02.285434 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:02.285876 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:02.285877 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:02.285929 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:02.286329 (+ 400us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:48:02.286334 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:48:02.286682 (+ 348us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:48:02.288162 (+ 1480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:48:02.288163 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:48:02.288164 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:48:02.288316 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:48:02.288469 (+ 153us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:48:02.288470 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:48:02.288470 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:48:02.288532 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:48:02.288733 (+ 201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:48:02.288734 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:48:02.288769 (+ 35us) client_negotiation.cc:769] Sending connection context
0807 08:48:02.288845 (+ 76us) client_negotiation.cc:240] Negotiation successful
0807 08:48:02.288944 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":81,"threads_started":1}
I20260807 08:48:02.289736 14371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:02.283248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44842 (local address 127.4.159.190:37483)
0807 08:48:02.283283 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:02.283285 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:02.283742 (+ 457us) server_negotiation.cc:407] Connection header received
0807 08:48:02.283798 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:02.283799 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:02.283809 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:02.283856 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:02.284186 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:02.284785 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:02.285471 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:02.285761 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:02.285821 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:02.286975 (+ 1154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:48:02.286977 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:48:02.286977 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:48:02.286983 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:48:02.288380 (+ 1397us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:48:02.288812 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:48:02.288813 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:48:02.288813 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:48:02.288870 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:48:02.289044 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:48:02.289045 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:48:02.289045 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:48:02.289163 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:48:02.289226 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0807 08:48:02.289482 (+ 256us) server_negotiation.cc:299] Negotiation successful
0807 08:48:02.289655 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 08:48:02.290041 14526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:02.283520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:37483 (local address 127.0.0.1:44842)
0807 08:48:02.283664 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:02.283675 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:02.283739 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:02.283873 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:02.283874 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:02.284042 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:48:02.284107 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:02.284109 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:02.284845 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:02.284846 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:02.285403 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:02.285405 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:02.285837 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:02.285838 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:02.285911 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:02.286419 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:48:02.286423 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:48:02.286852 (+ 429us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:48:02.288472 (+ 1620us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:48:02.288473 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:48:02.288473 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:48:02.288748 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:48:02.288939 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:48:02.288940 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:48:02.288940 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:48:02.288995 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:48:02.289304 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:48:02.289305 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:48:02.289347 (+ 42us) client_negotiation.cc:769] Sending connection context
0807 08:48:02.289854 (+ 507us) client_negotiation.cc:240] Negotiation successful
0807 08:48:02.289946 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":79,"threads_started":1}
Aug 07 08:48:02 dist-test-slave-6zbq krb5kdc[13584](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092480, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.159.190@KRBTEST.COM for oryx/127.4.159.189@KRBTEST.COM
I20260807 08:48:02.298448 14492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:02.291935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49204 (local address 127.4.159.189:37399)
0807 08:48:02.291971 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:02.291972 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:02.292117 (+ 145us) server_negotiation.cc:407] Connection header received
0807 08:48:02.292182 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:02.292183 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:02.292192 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:02.292255 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:02.292572 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:02.293281 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:02.293946 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:02.294230 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:02.294292 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:02.296109 (+ 1817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:48:02.296110 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:48:02.296110 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:48:02.296116 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:48:02.297303 (+ 1187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:48:02.297707 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:48:02.297707 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:48:02.297708 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:48:02.297803 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:48:02.297984 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:48:02.297985 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:48:02.297985 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:48:02.298101 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:48:02.298148 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0807 08:48:02.298317 (+ 169us) server_negotiation.cc:299] Negotiation successful
0807 08:48:02.298398 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260807 08:48:02.298532 14527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:02.291879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.189:37399 (local address 127.0.0.1:49204)
0807 08:48:02.292022 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:02.292062 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:02.292128 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:02.292258 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:02.292260 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:02.292476 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:48:02.292518 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:02.292519 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:02.293347 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:02.293348 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:02.293884 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:02.293885 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:02.294291 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:02.294291 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:02.294361 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:02.294767 (+ 406us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:48:02.294771 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:48:02.296034 (+ 1263us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:48:02.297370 (+ 1336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:48:02.297378 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:48:02.297380 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:48:02.297644 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:48:02.297863 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:48:02.297863 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:48:02.297864 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:48:02.297928 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:48:02.298154 (+ 226us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:48:02.298160 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:48:02.298212 (+ 52us) client_negotiation.cc:769] Sending connection context
0807 08:48:02.298293 (+ 81us) client_negotiation.cc:240] Negotiation successful
0807 08:48:02.298429 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":65,"threads_started":1}
I20260807 08:48:02.299042 13654 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.4.159.189:37399: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "aae8b08fd1ac4d2699d15a396e7ac8f2"
member_type: NON_VOTER
last_known_addr {
host: "127.4.159.189"
port: 37399
}
attrs {
promote: true
}
}
dest_uuid: "82a928c4f0234124b0567479c875bb25"
cas_config_opid_index: -1
I20260807 08:48:02.299326 13654 consensus_queue.cc:237] T 00000000000000000000000000000000 P 82a928c4f0234124b0567479c875bb25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "82a928c4f0234124b0567479c875bb25" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 37483 } } peers { permanent_uuid: "aae8b08fd1ac4d2699d15a396e7ac8f2" member_type: NON_VOTER last_known_addr { host: "127.4.159.189" port: 37399 } attrs { promote: true } }
I20260807 08:48:02.299878 14533 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 82a928c4f0234124b0567479c875bb25 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER aae8b08fd1ac4d2699d15a396e7ac8f2 (127.4.159.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "82a928c4f0234124b0567479c875bb25" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 37483 } } peers { permanent_uuid: "aae8b08fd1ac4d2699d15a396e7ac8f2" member_type: NON_VOTER last_known_addr { host: "127.4.159.189" port: 37399 } attrs { promote: true } } }
I20260807 08:48:02.300089 14444 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aae8b08fd1ac4d2699d15a396e7ac8f2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:48:02.300902 14444 raft_consensus.cc:1261] T 00000000000000000000000000000000 P aae8b08fd1ac4d2699d15a396e7ac8f2 [term 1 FOLLOWER]: Refusing update from remote peer 82a928c4f0234124b0567479c875bb25: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260807 08:48:02.300911 14533 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.4.159.189:37399
I20260807 08:48:02.301128 14532 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82a928c4f0234124b0567479c875bb25 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "82a928c4f0234124b0567479c875bb25" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "82a928c4f0234124b0567479c875bb25" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 37483 } } peers { permanent_uuid: "aae8b08fd1ac4d2699d15a396e7ac8f2" member_type: NON_VOTER last_known_addr { host: "127.4.159.189" port: 37399 } attrs { promote: true } } }
I20260807 08:48:02.301201 14532 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82a928c4f0234124b0567479c875bb25 [sys.catalog]: This master's current role is: LEADER
I20260807 08:48:02.301344 14532 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 82a928c4f0234124b0567479c875bb25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aae8b08fd1ac4d2699d15a396e7ac8f2" member_type: NON_VOTER last_known_addr { host: "127.4.159.189" port: 37399 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:48:02.301391 14377 master.cc:537] Master@127.4.159.189:37399 shutting down...
W20260807 08:48:02.302055 13632 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260807 08:48:02.302660 13632 consensus_peers.cc:487] T 00000000000000000000000000000000 P 82a928c4f0234124b0567479c875bb25 -> Peer aae8b08fd1ac4d2699d15a396e7ac8f2 (127.4.159.189:37399): Couldn't send request to peer aae8b08fd1ac4d2699d15a396e7ac8f2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260807 08:48:02.303962 14377 raft_consensus.cc:2229] T 00000000000000000000000000000000 P aae8b08fd1ac4d2699d15a396e7ac8f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:48:02.304049 14377 raft_consensus.cc:2258] T 00000000000000000000000000000000 P aae8b08fd1ac4d2699d15a396e7ac8f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:48:02.304097 14377 tablet_replica.cc:330] T 00000000000000000000000000000000 P aae8b08fd1ac4d2699d15a396e7ac8f2: stopping tablet replica
I20260807 08:48:02.316999 14377 master.cc:552] Master@127.4.159.189:37399 shutdown complete.
I20260807 08:48:02.317060 14377 master_runner.cc:305] Clearing existing system tablet
I20260807 08:48:02.317189 14377 fs_manager.cc:359] Metadata directory not provided
I20260807 08:48:02.317238 14377 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:48:02.317950 14377 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260807 08:48:02.318496 14539 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:48:02.318650 14377 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:48:02.318713 14377 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 08:48:02.318737 14377 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/master-1/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/master-1/wal
uuid: "aae8b08fd1ac4d2699d15a396e7ac8f2"
format_stamp: "Formatted at 2026-08-07 08:48:02 on dist-test-slave-6zbq"
server_key: "61793b2749c53d48bede6856776c4fc9"
server_key_iv: "caa4ba3f31e00156ac36bd53b861e153"
server_key_version: "encryptionkey@0"
I20260807 08:48:02.319624 14377 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P aae8b08fd1ac4d2699d15a396e7ac8f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:48:02.321067 14377 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P aae8b08fd1ac4d2699d15a396e7ac8f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260807 08:48:02.321156 14377 log.cc:1196] T 00000000000000000000000000000000 P aae8b08fd1ac4d2699d15a396e7ac8f2: Deleting WAL directory at /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260807 08:48:02.321352 14377 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P aae8b08fd1ac4d2699d15a396e7ac8f2: Deleting consensus metadata
I20260807 08:48:02.321484 14377 master_runner.cc:315] Copying system tablet from 127.4.159.190:37483
I20260807 08:48:02.322088 14377 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P aae8b08fd1ac4d2699d15a396e7ac8f2: tablet copy: Beginning tablet copy session from remote peer at address 127.4.159.190:37483
I20260807 08:48:02.327922 14371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:02.322484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44854 (local address 127.4.159.190:37483)
0807 08:48:02.322534 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:02.322536 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:02.322601 (+ 65us) server_negotiation.cc:407] Connection header received
0807 08:48:02.322686 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:02.322687 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:02.322695 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:02.322745 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:02.323079 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:02.323566 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:02.324583 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:02.324758 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:02.324802 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:02.325747 (+ 945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:48:02.325748 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:48:02.325748 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:48:02.325754 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:48:02.326981 (+ 1227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:48:02.327331 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:48:02.327331 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:48:02.327332 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:48:02.327377 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:48:02.327543 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:48:02.327543 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:48:02.327544 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:48:02.327633 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:48:02.327675 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0807 08:48:02.327808 (+ 133us) server_negotiation.cc:299] Negotiation successful
0807 08:48:02.327861 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260807 08:48:02.328001 14551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:02.322416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:37483 (local address 127.0.0.1:44854)
0807 08:48:02.322536 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:02.322547 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:02.322617 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:02.322752 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:02.322753 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:02.322949 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:48:02.323016 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:02.323017 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:02.323640 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:02.323640 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:02.324512 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:02.324515 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:02.324815 (+ 300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:02.324815 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:02.324883 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:02.325286 (+ 403us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:48:02.325291 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:48:02.325670 (+ 379us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:48:02.327050 (+ 1380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:48:02.327051 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:48:02.327052 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:48:02.327266 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:48:02.327427 (+ 161us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:48:02.327428 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:48:02.327428 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:48:02.327492 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:48:02.327682 (+ 190us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:48:02.327683 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:48:02.327728 (+ 45us) client_negotiation.cc:769] Sending connection context
0807 08:48:02.327792 (+ 64us) client_negotiation.cc:240] Negotiation successful
0807 08:48:02.327892 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":78,"threads_started":1}
I20260807 08:48:02.328384 13687 tablet_copy_service.cc:140] P 82a928c4f0234124b0567479c875bb25: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer aae8b08fd1ac4d2699d15a396e7ac8f2 ({username='oryx', principal='oryx/127.4.159.189@KRBTEST.COM'} at 127.0.0.1:44854)
I20260807 08:48:02.328495 13687 tablet_copy_service.cc:161] P 82a928c4f0234124b0567479c875bb25: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer aae8b08fd1ac4d2699d15a396e7ac8f2 at {username='oryx', principal='oryx/127.4.159.189@KRBTEST.COM'} at 127.0.0.1:44854: session id = aae8b08fd1ac4d2699d15a396e7ac8f2-00000000000000000000000000000000
I20260807 08:48:02.329142 13687 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 82a928c4f0234124b0567479c875bb25: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 08:48:02.329725 14377 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260807 08:48:02.330917 14377 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P aae8b08fd1ac4d2699d15a396e7ac8f2: tablet copy: Starting download of 0 data blocks...
I20260807 08:48:02.331053 14377 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P aae8b08fd1ac4d2699d15a396e7ac8f2: tablet copy: Starting download of 1 WAL segments...
I20260807 08:48:02.331910 14377 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P aae8b08fd1ac4d2699d15a396e7ac8f2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 08:48:02.332726 13687 tablet_copy_service.cc:342] P 82a928c4f0234124b0567479c875bb25: Request end of tablet copy session aae8b08fd1ac4d2699d15a396e7ac8f2-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.4.159.189@KRBTEST.COM'} at 127.0.0.1:44854
I20260807 08:48:02.332772 13687 tablet_copy_service.cc:433] P 82a928c4f0234124b0567479c875bb25: ending tablet copy session aae8b08fd1ac4d2699d15a396e7ac8f2-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer aae8b08fd1ac4d2699d15a396e7ac8f2
I20260807 08:48:02.335136 14377 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:48:02.336175 14554 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:48:02.336359 14556 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:48:02.336398 14553 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:48:02.336584 14377 server_base.cc:1034] running on GCE node
I20260807 08:48:02.336673 14377 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:48:02.336794 14377 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:48:02.337949 14377 hybrid_clock.cc:648] HybridClock initialized: now 1786092482337941 us; error 24 us; skew 500 ppm
Aug 07 08:48:02 dist-test-slave-6zbq krb5kdc[13584](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092482, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.159.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:48:02.339175 14377 init.cc:377] Logged in from keytab as oryx/127.4.159.189@KRBTEST.COM (short username oryx)
I20260807 08:48:02.339430 14377 webserver.cc:460] Webserver started at http://127.4.159.189:43459/ using document root <none> and password file <none>
I20260807 08:48:02.339540 14377 fs_manager.cc:359] Metadata directory not provided
I20260807 08:48:02.339574 14377 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:48:02.340212 14377 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260807 08:48:02.340714 14567 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:48:02.340849 14377 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 08:48:02.340894 14377 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/master-1/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/master-1/wal
uuid: "aae8b08fd1ac4d2699d15a396e7ac8f2"
format_stamp: "Formatted at 2026-08-07 08:48:02 on dist-test-slave-6zbq"
server_key: "61793b2749c53d48bede6856776c4fc9"
server_key_iv: "caa4ba3f31e00156ac36bd53b861e153"
server_key_version: "encryptionkey@0"
I20260807 08:48:02.340934 14377 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:48:02.349721 14377 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:48:02.351094 14377 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:48:02.353341 14377 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.189:37399
I20260807 08:48:02.353595 14652 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.189:37399 every 8 connection(s)
I20260807 08:48:02.353706 14377 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/master-1/data/info.pb
I20260807 08:48:02.354592 14655 sys_catalog.cc:263] Verifying existing consensus state
I20260807 08:48:02.355122 14655 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aae8b08fd1ac4d2699d15a396e7ac8f2: Bootstrap starting.
I20260807 08:48:02.358122 14655 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aae8b08fd1ac4d2699d15a396e7ac8f2: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 08:48:02.358417 14655 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aae8b08fd1ac4d2699d15a396e7ac8f2: Bootstrap complete.
I20260807 08:48:02.358644 14655 raft_consensus.cc:348] T 00000000000000000000000000000000 P aae8b08fd1ac4d2699d15a396e7ac8f2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "82a928c4f0234124b0567479c875bb25" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 37483 } } peers { permanent_uuid: "aae8b08fd1ac4d2699d15a396e7ac8f2" member_type: NON_VOTER last_known_addr { host: "127.4.159.189" port: 37399 } attrs { promote: true } }
I20260807 08:48:02.358736 14655 raft_consensus.cc:729] T 00000000000000000000000000000000 P aae8b08fd1ac4d2699d15a396e7ac8f2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: aae8b08fd1ac4d2699d15a396e7ac8f2, State: Initialized, Role: LEARNER
I20260807 08:48:02.358783 14655 consensus_queue.cc:260] T 00000000000000000000000000000000 P aae8b08fd1ac4d2699d15a396e7ac8f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "82a928c4f0234124b0567479c875bb25" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 37483 } } peers { permanent_uuid: "aae8b08fd1ac4d2699d15a396e7ac8f2" member_type: NON_VOTER last_known_addr { host: "127.4.159.189" port: 37399 } attrs { promote: true } }
I20260807 08:48:02.358959 14655 sys_catalog.cc:564] T 00000000000000000000000000000000 P aae8b08fd1ac4d2699d15a396e7ac8f2 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:48:02.358960 14659 sys_catalog.cc:455] T 00000000000000000000000000000000 P aae8b08fd1ac4d2699d15a396e7ac8f2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "82a928c4f0234124b0567479c875bb25" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 37483 } } peers { permanent_uuid: "aae8b08fd1ac4d2699d15a396e7ac8f2" member_type: NON_VOTER last_known_addr { host: "127.4.159.189" port: 37399 } attrs { promote: true } } }
I20260807 08:48:02.359066 14659 sys_catalog.cc:458] T 00000000000000000000000000000000 P aae8b08fd1ac4d2699d15a396e7ac8f2 [sys.catalog]: This master's current role is: LEARNER
I20260807 08:48:02.360468 14673 catalog_manager.cc:1259] Loaded cluster ID: 6e5ed93ced5c45e9b23a567a01055589
I20260807 08:48:02.360546 14673 catalog_manager.cc:1552] T 00000000000000000000000000000000 P aae8b08fd1ac4d2699d15a396e7ac8f2: loading cluster ID for follower catalog manager: success
I20260807 08:48:02.361577 14673 catalog_manager.cc:1574] T 00000000000000000000000000000000 P aae8b08fd1ac4d2699d15a396e7ac8f2: acquiring CA information for follower catalog manager: success
I20260807 08:48:02.361831 14673 catalog_manager.cc:1602] T 00000000000000000000000000000000 P aae8b08fd1ac4d2699d15a396e7ac8f2: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 08:48:02.376992 14674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:02.370454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49230 (local address 127.4.159.189:37399)
0807 08:48:02.370562 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:02.370564 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:02.370572 (+ 8us) server_negotiation.cc:407] Connection header received
0807 08:48:02.370649 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:02.370649 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:02.370658 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:02.370705 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:02.371337 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:02.371936 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:02.372967 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:02.373179 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:02.373232 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:02.374678 (+ 1446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:48:02.374680 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:48:02.374680 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:48:02.374709 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:48:02.375973 (+ 1264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:48:02.376353 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:48:02.376353 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:48:02.376354 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:48:02.376398 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:48:02.376579 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:48:02.376579 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:48:02.376580 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:48:02.376665 (+ 85us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:48:02.376737 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 08:48:02.376864 (+ 127us) server_negotiation.cc:299] Negotiation successful
0807 08:48:02.376919 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82,"thread_start_us":54,"threads_started":1}
WARNING: no policy specified for oryx/127.4.159.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.159.188@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.159.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/krb5kdc/oryx_127.4.159.188.keytab.
Entry for principal oryx/127.4.159.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/krb5kdc/oryx_127.4.159.188.keytab.
I20260807 08:48:02.401010 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.159.188:35089
--webserver_interface=127.4.159.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/krb5kdc/oryx_127.4.159.188.keytab
--principal=oryx/127.4.159.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:45003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.159.190:37483,127.4.159.189:37399,127.4.159.188:35089
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:48:02.481772 14677 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:48:02.481954 14677 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:48:02.481974 14677 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:48:02.483088 14677 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:48:02.483126 14677 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:48:02.483139 14677 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:48:02.483150 14677 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:48:02.483165 14677 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:48:02.484575 14677 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:45003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.4.159.190:37483,127.4.159.189:37399,127.4.159.188:35089
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/krb5kdc/oryx_127.4.159.188.keytab
--principal=oryx/127.4.159.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.159.188:35089
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.4.159.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14677
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:48:02.484787 14677 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:48:02.484958 14677 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:48:02.489152 14684 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:48:02.489182 14689 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:48:02.489228 14686 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:48:02.489357 14677 server_base.cc:1034] running on GCE node
I20260807 08:48:02.489555 14677 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:48:02.489796 14677 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:48:02.490917 14677 hybrid_clock.cc:648] HybridClock initialized: now 1786092482490908 us; error 31 us; skew 500 ppm
Aug 07 08:48:02 dist-test-slave-6zbq krb5kdc[13584](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092482, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.159.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:48:02.493216 14677 init.cc:377] Logged in from keytab as oryx/127.4.159.188@KRBTEST.COM (short username oryx)
I20260807 08:48:02.493606 14677 webserver.cc:460] Webserver started at http://127.4.159.188:38419/ using document root <none> and password file <none>
I20260807 08:48:02.493811 14677 fs_manager.cc:359] Metadata directory not provided
I20260807 08:48:02.493856 14677 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:48:02.493975 14677 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:48:02.494959 14677 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/master-2/data/instance:
uuid: "383bca56225440c4966e85359729cd9a"
format_stamp: "Formatted at 2026-08-07 08:48:02 on dist-test-slave-6zbq"
server_key: "0607f38541869517f89c1da5d0ea29c9"
server_key_iv: "bcb5d6fdd8c3d8677d0ab1c3fef6b41b"
server_key_version: "encryptionkey@0"
I20260807 08:48:02.495272 14677 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/master-2/wal/instance:
uuid: "383bca56225440c4966e85359729cd9a"
format_stamp: "Formatted at 2026-08-07 08:48:02 on dist-test-slave-6zbq"
server_key: "0607f38541869517f89c1da5d0ea29c9"
server_key_iv: "bcb5d6fdd8c3d8677d0ab1c3fef6b41b"
server_key_version: "encryptionkey@0"
I20260807 08:48:02.496472 14677 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 08:48:02.497143 14698 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:48:02.497332 14677 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 08:48:02.497448 14677 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/master-2/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/master-2/wal
uuid: "383bca56225440c4966e85359729cd9a"
format_stamp: "Formatted at 2026-08-07 08:48:02 on dist-test-slave-6zbq"
server_key: "0607f38541869517f89c1da5d0ea29c9"
server_key_iv: "bcb5d6fdd8c3d8677d0ab1c3fef6b41b"
server_key_version: "encryptionkey@0"
I20260807 08:48:02.497521 14677 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:48:02.505573 14677 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:48:02.507486 14677 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:48:02.507612 14677 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:48:02.511365 14677 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.188:35089
I20260807 08:48:02.511392 14787 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.188:35089 every 8 connection(s)
I20260807 08:48:02.511689 14677 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/master-2/data/info.pb
I20260807 08:48:02.512573 14788 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:48:02.513908 14788 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.159.190" port: 37483 } has no permanent_uuid. Determining permanent_uuid...
I20260807 08:48:02.515170 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 14677
Aug 07 08:48:02 dist-test-slave-6zbq krb5kdc[13584](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092482, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.159.188@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM
Aug 07 08:48:02 dist-test-slave-6zbq krb5kdc[13584](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092479, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.4.159.188@KRBTEST.COM
I20260807 08:48:02.523319 14371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:02.514542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44868 (local address 127.4.159.190:37483)
0807 08:48:02.514602 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:02.514604 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:02.514766 (+ 162us) server_negotiation.cc:407] Connection header received
0807 08:48:02.515205 (+ 439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:02.515206 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:02.515219 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:02.515271 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:02.515922 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:02.516429 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:02.517429 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:02.517615 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:02.517665 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:02.520611 (+ 2946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:48:02.520612 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:48:02.520612 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:48:02.520620 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:48:02.521988 (+ 1368us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:48:02.522502 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:48:02.522502 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:48:02.522503 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:48:02.522552 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:48:02.522785 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:48:02.522786 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:48:02.522786 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:48:02.522887 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:48:02.522950 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0807 08:48:02.523171 (+ 221us) server_negotiation.cc:299] Negotiation successful
0807 08:48:02.523232 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260807 08:48:02.523478 14792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:02.514470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:37483 (local address 127.0.0.1:44868)
0807 08:48:02.514660 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:02.514692 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:02.515108 (+ 416us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:02.515340 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:02.515344 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:02.515595 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:48:02.515843 (+ 248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:02.515847 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:02.516529 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:02.516530 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:02.517345 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:02.517347 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:02.517697 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:02.517698 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:02.517824 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:02.519129 (+ 1305us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:48:02.519138 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:48:02.520533 (+ 1395us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:48:02.522065 (+ 1532us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:48:02.522073 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:48:02.522075 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:48:02.522436 (+ 361us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:48:02.522610 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:48:02.522611 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:48:02.522611 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:48:02.522729 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:48:02.522946 (+ 217us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:48:02.522950 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:48:02.523049 (+ 99us) client_negotiation.cc:769] Sending connection context
0807 08:48:02.523137 (+ 88us) client_negotiation.cc:240] Negotiation successful
0807 08:48:02.523289 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":100,"threads_started":1}
I20260807 08:48:02.524006 14793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:02.515757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57350 (local address 127.4.159.188:35089)
0807 08:48:02.515890 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:02.515894 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:02.515908 (+ 14us) server_negotiation.cc:407] Connection header received
0807 08:48:02.515926 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:02.515927 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:02.515947 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:02.516008 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:02.516566 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:02.517308 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:02.518496 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:02.518846 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:02.518902 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:02.521132 (+ 2230us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:48:02.521139 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:48:02.521139 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:48:02.521146 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:48:02.522825 (+ 1679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:48:02.523131 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:48:02.523134 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:48:02.523134 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:48:02.523195 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:48:02.523377 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:48:02.523377 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:48:02.523378 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:48:02.523526 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:48:02.523797 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0807 08:48:02.523841 (+ 44us) server_negotiation.cc:299] Negotiation successful
0807 08:48:02.523914 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":69,"threads_started":1}
I20260807 08:48:02.524080 14788 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.159.189" port: 37399 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 08:48:02 dist-test-slave-6zbq krb5kdc[13584](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092482, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.159.188@KRBTEST.COM for oryx/127.4.159.189@KRBTEST.COM
I20260807 08:48:02.528970 14792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:02.524286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.189:37399 (local address 127.0.0.1:49238)
0807 08:48:02.524327 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:02.524338 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:02.524378 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:02.524506 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:02.524506 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:02.524621 (+ 115us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:48:02.524655 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:02.524656 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:02.525210 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:02.525210 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:02.525666 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:02.525667 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:02.525929 (+ 262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:02.525930 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:02.525967 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:02.526228 (+ 261us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:48:02.526231 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:48:02.527215 (+ 984us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:48:02.528340 (+ 1125us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:48:02.528341 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:48:02.528341 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:48:02.528483 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:48:02.528607 (+ 124us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:48:02.528608 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:48:02.528608 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:48:02.528654 (+ 46us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:48:02.528798 (+ 144us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:48:02.528798 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:48:02.528825 (+ 27us) client_negotiation.cc:769] Sending connection context
0807 08:48:02.528863 (+ 38us) client_negotiation.cc:240] Negotiation successful
0807 08:48:02.528918 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260807 08:48:02.528995 14674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:02.524342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49238 (local address 127.4.159.189:37399)
0807 08:48:02.524381 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:02.524382 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:02.524406 (+ 24us) server_negotiation.cc:407] Connection header received
0807 08:48:02.524431 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:02.524432 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:02.524443 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:02.524487 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:02.524710 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:02.525157 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:02.525716 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:02.525886 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:02.525924 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:02.527287 (+ 1363us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:48:02.527288 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:48:02.527288 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:48:02.527294 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:48:02.528272 (+ 978us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:48:02.528531 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:48:02.528532 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:48:02.528532 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:48:02.528573 (+ 41us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:48:02.528685 (+ 112us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:48:02.528685 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:48:02.528686 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:48:02.528765 (+ 79us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:48:02.528805 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0807 08:48:02.528885 (+ 80us) server_negotiation.cc:299] Negotiation successful
0807 08:48:02.528939 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260807 08:48:02.529328 14788 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.159.188" port: 35089 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 08:48:02 dist-test-slave-6zbq krb5kdc[13584](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092482, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.159.188@KRBTEST.COM for oryx/127.4.159.188@KRBTEST.COM
I20260807 08:48:02.534255 14792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:02.529514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.188:35089 (local address 127.0.0.1:57358)
0807 08:48:02.529557 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:02.529566 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:02.529600 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:02.529679 (+ 79us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:02.529679 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:02.529775 (+ 96us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:48:02.529799 (+ 24us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:02.529799 (+ 0us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:02.530330 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:02.530330 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:02.530803 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:02.530805 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:02.531091 (+ 286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:02.531091 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:02.531122 (+ 31us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:02.531392 (+ 270us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:48:02.531395 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:48:02.532348 (+ 953us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:48:02.533616 (+ 1268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:48:02.533617 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:48:02.533617 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:48:02.533763 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:48:02.533890 (+ 127us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:48:02.533890 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:48:02.533891 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:48:02.533929 (+ 38us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:48:02.534072 (+ 143us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:48:02.534072 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:48:02.534100 (+ 28us) client_negotiation.cc:769] Sending connection context
0807 08:48:02.534139 (+ 39us) client_negotiation.cc:240] Negotiation successful
0807 08:48:02.534202 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260807 08:48:02.534330 14793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:02.529564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57358 (local address 127.4.159.188:35089)
0807 08:48:02.529596 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:02.529598 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:02.529605 (+ 7us) server_negotiation.cc:407] Connection header received
0807 08:48:02.529628 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:02.529629 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:02.529637 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:02.529675 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:02.529841 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:02.530272 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:02.530865 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:02.531046 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:02.531086 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:02.532413 (+ 1327us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:48:02.532414 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:48:02.532414 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:48:02.532419 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:48:02.533552 (+ 1133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:48:02.533809 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:48:02.533810 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:48:02.533810 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:48:02.533850 (+ 40us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:48:02.533961 (+ 111us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:48:02.533961 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:48:02.533961 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:48:02.534034 (+ 73us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:48:02.534074 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0807 08:48:02.534160 (+ 86us) server_negotiation.cc:299] Negotiation successful
0807 08:48:02.534243 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":31,"server-negotiator.queue_time_us":16}
I20260807 08:48:02.535344 14788 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 383bca56225440c4966e85359729cd9a: Bootstrap starting.
I20260807 08:48:02.536048 14788 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 383bca56225440c4966e85359729cd9a: Neither blocks nor log segments found. Creating new log.
I20260807 08:48:02.536294 14788 log.cc:824] T 00000000000000000000000000000000 P 383bca56225440c4966e85359729cd9a: Log is configured to *not* fsync() on all Append() calls
I20260807 08:48:02.536880 14788 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 383bca56225440c4966e85359729cd9a: No bootstrap required, opened a new log
I20260807 08:48:02.538165 14788 raft_consensus.cc:348] T 00000000000000000000000000000000 P 383bca56225440c4966e85359729cd9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82a928c4f0234124b0567479c875bb25" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 37483 } } peers { permanent_uuid: "aae8b08fd1ac4d2699d15a396e7ac8f2" member_type: VOTER last_known_addr { host: "127.4.159.189" port: 37399 } } peers { permanent_uuid: "383bca56225440c4966e85359729cd9a" member_type: VOTER last_known_addr { host: "127.4.159.188" port: 35089 } }
I20260807 08:48:02.538270 14788 raft_consensus.cc:374] T 00000000000000000000000000000000 P 383bca56225440c4966e85359729cd9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:48:02.538301 14788 raft_consensus.cc:729] T 00000000000000000000000000000000 P 383bca56225440c4966e85359729cd9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 383bca56225440c4966e85359729cd9a, State: Initialized, Role: FOLLOWER
I20260807 08:48:02.538446 14788 consensus_queue.cc:260] T 00000000000000000000000000000000 P 383bca56225440c4966e85359729cd9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82a928c4f0234124b0567479c875bb25" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 37483 } } peers { permanent_uuid: "aae8b08fd1ac4d2699d15a396e7ac8f2" member_type: VOTER last_known_addr { host: "127.4.159.189" port: 37399 } } peers { permanent_uuid: "383bca56225440c4966e85359729cd9a" member_type: VOTER last_known_addr { host: "127.4.159.188" port: 35089 } }
I20260807 08:48:02.538699 14788 sys_catalog.cc:564] T 00000000000000000000000000000000 P 383bca56225440c4966e85359729cd9a [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:48:02.538676 14795 sys_catalog.cc:455] T 00000000000000000000000000000000 P 383bca56225440c4966e85359729cd9a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82a928c4f0234124b0567479c875bb25" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 37483 } } peers { permanent_uuid: "aae8b08fd1ac4d2699d15a396e7ac8f2" member_type: VOTER last_known_addr { host: "127.4.159.189" port: 37399 } } peers { permanent_uuid: "383bca56225440c4966e85359729cd9a" member_type: VOTER last_known_addr { host: "127.4.159.188" port: 35089 } } }
I20260807 08:48:02.538741 14795 sys_catalog.cc:458] T 00000000000000000000000000000000 P 383bca56225440c4966e85359729cd9a [sys.catalog]: This master's current role is: FOLLOWER
W20260807 08:48:02.540179 14812 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 383bca56225440c4966e85359729cd9a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 08:48:02.540230 14812 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 08:48:02.542104 14677 master_runner.cc:428] Detected that this master 383bca56225440c4966e85359729cd9a is joining an existing cluster
I20260807 08:48:02.542161 14677 master_runner.cc:432] Initiating AddMaster RPC to add 127.4.159.188:35089
I20260807 08:48:02.549638 14371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:02.543528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44874 (local address 127.4.159.190:37483)
0807 08:48:02.543591 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:02.543592 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:02.543845 (+ 253us) server_negotiation.cc:407] Connection header received
0807 08:48:02.543887 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:02.543888 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:02.543897 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:02.543974 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:02.544271 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:02.544994 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:02.546123 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:02.546422 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:02.546463 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:02.547503 (+ 1040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:48:02.547504 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:48:02.547504 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:48:02.547510 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:48:02.548675 (+ 1165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:48:02.549018 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:48:02.549018 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:48:02.549018 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:48:02.549059 (+ 41us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:48:02.549247 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:48:02.549248 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:48:02.549248 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:48:02.549348 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:48:02.549398 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0807 08:48:02.549528 (+ 130us) server_negotiation.cc:299] Negotiation successful
0807 08:48:02.549582 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 08:48:02.549753 14825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:02.543427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:37483 (local address 127.0.0.1:44874)
0807 08:48:02.543776 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:02.543785 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:02.543837 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:02.543980 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:02.543980 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:02.544153 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:48:02.544211 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:02.544212 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:02.545148 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:02.545149 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:02.546014 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:02.546017 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:02.546532 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:02.546532 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:02.546594 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:02.547006 (+ 412us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:48:02.547010 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:48:02.547412 (+ 402us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:48:02.548751 (+ 1339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:48:02.548752 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:48:02.548753 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:48:02.548952 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:48:02.549118 (+ 166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:48:02.549118 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:48:02.549118 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:48:02.549177 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:48:02.549405 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:48:02.549406 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:48:02.549446 (+ 40us) client_negotiation.cc:769] Sending connection context
0807 08:48:02.549511 (+ 65us) client_negotiation.cc:240] Negotiation successful
0807 08:48:02.549601 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":76,"threads_started":1}
Aug 07 08:48:02 dist-test-slave-6zbq krb5kdc[13584](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092480, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.159.190@KRBTEST.COM for oryx/127.4.159.188@KRBTEST.COM
I20260807 08:48:02.569243 14527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:02.551396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.188:35089 (local address 127.0.0.1:57366)
0807 08:48:02.551561 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:02.551570 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:02.551617 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:02.551748 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:02.551749 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:02.551900 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:48:02.551936 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:02.551937 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:02.552499 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:02.552500 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:02.553291 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:02.553293 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:02.553592 (+ 299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:02.553593 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:02.553654 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:02.553941 (+ 287us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:48:02.553946 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:48:02.555156 (+ 1210us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:48:02.568249 (+ 13093us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:48:02.568250 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:48:02.568250 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:48:02.568454 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:48:02.568657 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:48:02.568657 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:48:02.568658 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:48:02.568712 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:48:02.568950 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:48:02.568950 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:48:02.568989 (+ 39us) client_negotiation.cc:769] Sending connection context
0807 08:48:02.569032 (+ 43us) client_negotiation.cc:240] Negotiation successful
0807 08:48:02.569147 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148}
I20260807 08:48:02.569711 14832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:02.551457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57366 (local address 127.4.159.188:35089)
0807 08:48:02.551570 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:02.551571 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:02.551603 (+ 32us) server_negotiation.cc:407] Connection header received
0807 08:48:02.551681 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:02.551682 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:02.551697 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:02.551744 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:02.551999 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:02.552441 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:02.553353 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:02.553540 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:02.553580 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:02.555248 (+ 1668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:48:02.555249 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:48:02.555249 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:48:02.555255 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:48:02.568178 (+ 12923us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:48:02.568535 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:48:02.568536 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:48:02.568536 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:48:02.568587 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:48:02.568774 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:48:02.568775 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:48:02.568775 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:48:02.568884 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:48:02.569498 (+ 614us) server_negotiation.cc:1035] Waiting for connection context
0807 08:48:02.569538 (+ 40us) server_negotiation.cc:299] Negotiation successful
0807 08:48:02.569612 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":68,"threads_started":1}
I20260807 08:48:02.570194 13654 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.4.159.188:35089: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "383bca56225440c4966e85359729cd9a"
member_type: NON_VOTER
last_known_addr {
host: "127.4.159.188"
port: 35089
}
attrs {
promote: true
}
}
dest_uuid: "82a928c4f0234124b0567479c875bb25"
cas_config_opid_index: 5
I20260807 08:48:02.570297 14831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:02.543620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.188:35089 (local address 127.0.0.1:57362)
0807 08:48:02.544324 (+ 704us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:02.544332 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:02.544395 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:02.544517 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:02.544518 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:02.544695 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:48:02.544743 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:02.544745 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:02.545518 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:02.545519 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:02.546259 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:02.546260 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:02.546675 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:02.546675 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:02.546729 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:02.566795 (+ 20066us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:48:02.566799 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:48:02.567207 (+ 408us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:48:02.569158 (+ 1951us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:48:02.569159 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:48:02.569159 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:48:02.569498 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:48:02.569661 (+ 163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:48:02.569662 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:48:02.569662 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:48:02.569708 (+ 46us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:48:02.570005 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:48:02.570005 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:48:02.570043 (+ 38us) client_negotiation.cc:769] Sending connection context
0807 08:48:02.570105 (+ 62us) client_negotiation.cc:240] Negotiation successful
0807 08:48:02.570203 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":687,"thread_start_us":74,"threads_started":1}
I20260807 08:48:02.570457 13654 consensus_queue.cc:237] T 00000000000000000000000000000000 P 82a928c4f0234124b0567479c875bb25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "82a928c4f0234124b0567479c875bb25" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 37483 } } peers { permanent_uuid: "aae8b08fd1ac4d2699d15a396e7ac8f2" member_type: NON_VOTER last_known_addr { host: "127.4.159.189" port: 37399 } attrs { promote: true } } peers { permanent_uuid: "383bca56225440c4966e85359729cd9a" member_type: NON_VOTER last_known_addr { host: "127.4.159.188" port: 35089 } attrs { promote: true } }
I20260807 08:48:02.570585 14793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:02.543661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57362 (local address 127.4.159.188:35089)
0807 08:48:02.543692 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:02.543693 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:02.544368 (+ 675us) server_negotiation.cc:407] Connection header received
0807 08:48:02.544447 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:02.544447 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:02.544454 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:02.544477 (+ 23us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:02.544810 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:02.545441 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:02.546327 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:02.546583 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:02.546619 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:02.567286 (+ 20667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:48:02.567287 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:48:02.567287 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:48:02.567293 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:48:02.569082 (+ 1789us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:48:02.569561 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:48:02.569562 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:48:02.569562 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:48:02.569606 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:48:02.569761 (+ 155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:48:02.569762 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:48:02.569762 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:48:02.569856 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:48:02.569904 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0807 08:48:02.570455 (+ 551us) server_negotiation.cc:299] Negotiation successful
0807 08:48:02.570520 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":18}
I20260807 08:48:02.570809 14533 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 82a928c4f0234124b0567479c875bb25 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 383bca56225440c4966e85359729cd9a (127.4.159.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "82a928c4f0234124b0567479c875bb25" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 37483 } } peers { permanent_uuid: "aae8b08fd1ac4d2699d15a396e7ac8f2" member_type: NON_VOTER last_known_addr { host: "127.4.159.189" port: 37399 } attrs { promote: true } } peers { permanent_uuid: "383bca56225440c4966e85359729cd9a" member_type: NON_VOTER last_known_addr { host: "127.4.159.188" port: 35089 } attrs { promote: true } } }
I20260807 08:48:02.570995 14674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:02.543490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49242 (local address 127.4.159.189:37399)
0807 08:48:02.543563 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:02.543565 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:02.543685 (+ 120us) server_negotiation.cc:407] Connection header received
0807 08:48:02.543778 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:02.543779 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:02.543790 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:02.543818 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:02.544298 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:02.545005 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:02.546082 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:02.546386 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:02.546421 (+ 35us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:02.567547 (+ 21126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:48:02.567549 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:48:02.567549 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:48:02.567555 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:48:02.569049 (+ 1494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:48:02.569481 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:48:02.569482 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:48:02.569482 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:48:02.569537 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:48:02.569736 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:48:02.569737 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:48:02.569737 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:48:02.569852 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:48:02.569892 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0807 08:48:02.570837 (+ 945us) server_negotiation.cc:299] Negotiation successful
0807 08:48:02.570913 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 08:48:02.571267 14827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:02.543423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.189:37399 (local address 127.0.0.1:49242)
0807 08:48:02.543587 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:02.543636 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:02.543710 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:02.543877 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:02.543878 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:02.544067 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:48:02.544204 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:02.544206 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:02.545105 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:02.545106 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:02.545995 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:02.545998 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:02.546496 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:02.546496 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:02.546558 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:02.567067 (+ 20509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:48:02.567072 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:48:02.567448 (+ 376us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:48:02.569137 (+ 1689us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:48:02.569138 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:48:02.569139 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:48:02.569395 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:48:02.569605 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:48:02.569605 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:48:02.569605 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:48:02.569668 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:48:02.570059 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:48:02.570059 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:48:02.570092 (+ 33us) client_negotiation.cc:769] Sending connection context
0807 08:48:02.571065 (+ 973us) client_negotiation.cc:240] Negotiation successful
0807 08:48:02.571174 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"mutex_wait_us":38,"thread_start_us":93,"threads_started":1}
I20260807 08:48:02.571653 14533 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.4.159.188:35089
I20260807 08:48:02.571731 14533 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82a928c4f0234124b0567479c875bb25 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "82a928c4f0234124b0567479c875bb25" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "82a928c4f0234124b0567479c875bb25" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 37483 } } peers { permanent_uuid: "aae8b08fd1ac4d2699d15a396e7ac8f2" member_type: NON_VOTER last_known_addr { host: "127.4.159.189" port: 37399 } attrs { promote: true } } peers { permanent_uuid: "383bca56225440c4966e85359729cd9a" member_type: NON_VOTER last_known_addr { host: "127.4.159.188" port: 35089 } attrs { promote: true } } }
I20260807 08:48:02.571800 14533 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82a928c4f0234124b0567479c875bb25 [sys.catalog]: This master's current role is: LEADER
I20260807 08:48:02.572278 14677 master.cc:537] Master@127.4.159.188:35089 shutting down...
W20260807 08:48:02.573189 13628 consensus_peers.cc:487] T 00000000000000000000000000000000 P 82a928c4f0234124b0567479c875bb25 -> Peer 383bca56225440c4966e85359729cd9a (127.4.159.188:35089): Couldn't send request to peer 383bca56225440c4966e85359729cd9a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260807 08:48:02.575172 14677 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 383bca56225440c4966e85359729cd9a [term 0 FOLLOWER]: Raft consensus shutting down.
I20260807 08:48:02.575255 14677 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 383bca56225440c4966e85359729cd9a [term 0 FOLLOWER]: Raft consensus is shut down!
I20260807 08:48:02.575299 14677 tablet_replica.cc:330] T 00000000000000000000000000000000 P 383bca56225440c4966e85359729cd9a: stopping tablet replica
I20260807 08:48:02.575383 14527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:02.571957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.189:37399 (local address 127.0.0.1:49252)
0807 08:48:02.572000 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:02.572010 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:02.572052 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:02.572235 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:02.572235 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:02.572237 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:48:02.572267 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:02.572268 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:02.573186 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:02.573187 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:02.574297 (+ 1110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:02.574300 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:02.575159 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:02.575160 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:02.575221 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:02.575241 (+ 20us) client_negotiation.cc:769] Sending connection context
0807 08:48:02.575293 (+ 52us) client_negotiation.cc:240] Negotiation successful
0807 08:48:02.575326 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260807 08:48:02.575595 14674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:02.572004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49252 (local address 127.4.159.189:37399)
0807 08:48:02.572035 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:02.572037 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:02.572044 (+ 7us) server_negotiation.cc:407] Connection header received
0807 08:48:02.572115 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:02.572116 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:02.572137 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:02.572185 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:48:02.572418 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:02.573096 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:02.574434 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:02.574989 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:02.575052 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:02.575123 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 08:48:02.575508 (+ 385us) server_negotiation.cc:299] Negotiation successful
0807 08:48:02.575540 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260807 08:48:02.575898 14616 raft_consensus.cc:1261] T 00000000000000000000000000000000 P aae8b08fd1ac4d2699d15a396e7ac8f2 [term 1 LEARNER]: Refusing update from remote peer 82a928c4f0234124b0567479c875bb25: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260807 08:48:02.576092 14532 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 82a928c4f0234124b0567479c875bb25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aae8b08fd1ac4d2699d15a396e7ac8f2" member_type: NON_VOTER last_known_addr { host: "127.4.159.189" port: 37399 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260807 08:48:02.576642 14616 raft_consensus.cc:2932] T 00000000000000000000000000000000 P aae8b08fd1ac4d2699d15a396e7ac8f2 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 383bca56225440c4966e85359729cd9a (127.4.159.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "82a928c4f0234124b0567479c875bb25" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 37483 } } peers { permanent_uuid: "aae8b08fd1ac4d2699d15a396e7ac8f2" member_type: NON_VOTER last_known_addr { host: "127.4.159.189" port: 37399 } attrs { promote: true } } peers { permanent_uuid: "383bca56225440c4966e85359729cd9a" member_type: NON_VOTER last_known_addr { host: "127.4.159.188" port: 35089 } attrs { promote: true } } }
I20260807 08:48:02.577719 14835 sys_catalog.cc:455] T 00000000000000000000000000000000 P aae8b08fd1ac4d2699d15a396e7ac8f2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 82a928c4f0234124b0567479c875bb25. Latest consensus state: current_term: 1 leader_uuid: "82a928c4f0234124b0567479c875bb25" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "82a928c4f0234124b0567479c875bb25" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 37483 } } peers { permanent_uuid: "aae8b08fd1ac4d2699d15a396e7ac8f2" member_type: NON_VOTER last_known_addr { host: "127.4.159.189" port: 37399 } attrs { promote: true } } peers { permanent_uuid: "383bca56225440c4966e85359729cd9a" member_type: NON_VOTER last_known_addr { host: "127.4.159.188" port: 35089 } attrs { promote: true } } }
I20260807 08:48:02.577793 14835 sys_catalog.cc:458] T 00000000000000000000000000000000 P aae8b08fd1ac4d2699d15a396e7ac8f2 [sys.catalog]: This master's current role is: LEARNER
I20260807 08:48:02.577808 14659 sys_catalog.cc:455] T 00000000000000000000000000000000 P aae8b08fd1ac4d2699d15a396e7ac8f2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "82a928c4f0234124b0567479c875bb25" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "82a928c4f0234124b0567479c875bb25" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 37483 } } peers { permanent_uuid: "aae8b08fd1ac4d2699d15a396e7ac8f2" member_type: NON_VOTER last_known_addr { host: "127.4.159.189" port: 37399 } attrs { promote: true } } peers { permanent_uuid: "383bca56225440c4966e85359729cd9a" member_type: NON_VOTER last_known_addr { host: "127.4.159.188" port: 35089 } attrs { promote: true } } }
I20260807 08:48:02.577891 14659 sys_catalog.cc:458] T 00000000000000000000000000000000 P aae8b08fd1ac4d2699d15a396e7ac8f2 [sys.catalog]: This master's current role is: LEARNER
I20260807 08:48:02.578603 14532 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 82a928c4f0234124b0567479c875bb25: attempting to promote NON_VOTER aae8b08fd1ac4d2699d15a396e7ac8f2 to VOTER
I20260807 08:48:02.578756 14532 consensus_queue.cc:237] T 00000000000000000000000000000000 P 82a928c4f0234124b0567479c875bb25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "82a928c4f0234124b0567479c875bb25" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 37483 } } peers { permanent_uuid: "aae8b08fd1ac4d2699d15a396e7ac8f2" member_type: VOTER last_known_addr { host: "127.4.159.189" port: 37399 } attrs { promote: false } } peers { permanent_uuid: "383bca56225440c4966e85359729cd9a" member_type: NON_VOTER last_known_addr { host: "127.4.159.188" port: 35089 } attrs { promote: true } }
I20260807 08:48:02.579172 14527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:02.579082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.188:35089 (local address 127.0.0.1:57374)
0807 08:48:02.579111 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:02.579148 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.188:35089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
I20260807 08:48:02.579262 14616 raft_consensus.cc:1261] T 00000000000000000000000000000000 P aae8b08fd1ac4d2699d15a396e7ac8f2 [term 1 LEARNER]: Refusing update from remote peer 82a928c4f0234124b0567479c875bb25: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260807 08:48:02.579437 14533 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 82a928c4f0234124b0567479c875bb25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aae8b08fd1ac4d2699d15a396e7ac8f2" member_type: VOTER last_known_addr { host: "127.4.159.189" port: 37399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260807 08:48:02.579484 14527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:02.579405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.188:35089 (local address 127.0.0.1:57388)
0807 08:48:02.579438 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:02.579466 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.188:35089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
W20260807 08:48:02.579546 13628 consensus_peers.cc:487] T 00000000000000000000000000000000 P 82a928c4f0234124b0567479c875bb25 -> Peer 383bca56225440c4966e85359729cd9a (127.4.159.188:35089): Couldn't send request to peer 383bca56225440c4966e85359729cd9a. Status: Network error: Client connection negotiation failed: client connection to 127.4.159.188:35089: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260807 08:48:02.580008 14532 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 82a928c4f0234124b0567479c875bb25 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, aae8b08fd1ac4d2699d15a396e7ac8f2 (127.4.159.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "82a928c4f0234124b0567479c875bb25" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 37483 } } peers { permanent_uuid: "aae8b08fd1ac4d2699d15a396e7ac8f2" member_type: VOTER last_known_addr { host: "127.4.159.189" port: 37399 } attrs { promote: false } } peers { permanent_uuid: "383bca56225440c4966e85359729cd9a" member_type: NON_VOTER last_known_addr { host: "127.4.159.188" port: 35089 } attrs { promote: true } } }
I20260807 08:48:02.580644 14532 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82a928c4f0234124b0567479c875bb25 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "82a928c4f0234124b0567479c875bb25" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "82a928c4f0234124b0567479c875bb25" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 37483 } } peers { permanent_uuid: "aae8b08fd1ac4d2699d15a396e7ac8f2" member_type: VOTER last_known_addr { host: "127.4.159.189" port: 37399 } attrs { promote: false } } peers { permanent_uuid: "383bca56225440c4966e85359729cd9a" member_type: NON_VOTER last_known_addr { host: "127.4.159.188" port: 35089 } attrs { promote: true } } }
I20260807 08:48:02.580677 14533 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82a928c4f0234124b0567479c875bb25 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "82a928c4f0234124b0567479c875bb25" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "82a928c4f0234124b0567479c875bb25" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 37483 } } peers { permanent_uuid: "aae8b08fd1ac4d2699d15a396e7ac8f2" member_type: VOTER last_known_addr { host: "127.4.159.189" port: 37399 } attrs { promote: false } } peers { permanent_uuid: "383bca56225440c4966e85359729cd9a" member_type: NON_VOTER last_known_addr { host: "127.4.159.188" port: 35089 } attrs { promote: true } } }
I20260807 08:48:02.580724 14532 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82a928c4f0234124b0567479c875bb25 [sys.catalog]: This master's current role is: LEADER
I20260807 08:48:02.580727 14533 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82a928c4f0234124b0567479c875bb25 [sys.catalog]: This master's current role is: LEADER
I20260807 08:48:02.580951 14616 raft_consensus.cc:2932] T 00000000000000000000000000000000 P aae8b08fd1ac4d2699d15a396e7ac8f2 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, aae8b08fd1ac4d2699d15a396e7ac8f2 (127.4.159.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "82a928c4f0234124b0567479c875bb25" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 37483 } } peers { permanent_uuid: "aae8b08fd1ac4d2699d15a396e7ac8f2" member_type: VOTER last_known_addr { host: "127.4.159.189" port: 37399 } attrs { promote: false } } peers { permanent_uuid: "383bca56225440c4966e85359729cd9a" member_type: NON_VOTER last_known_addr { host: "127.4.159.188" port: 35089 } attrs { promote: true } } }
I20260807 08:48:02.581552 14659 sys_catalog.cc:455] T 00000000000000000000000000000000 P aae8b08fd1ac4d2699d15a396e7ac8f2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "82a928c4f0234124b0567479c875bb25" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "82a928c4f0234124b0567479c875bb25" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 37483 } } peers { permanent_uuid: "aae8b08fd1ac4d2699d15a396e7ac8f2" member_type: VOTER last_known_addr { host: "127.4.159.189" port: 37399 } attrs { promote: false } } peers { permanent_uuid: "383bca56225440c4966e85359729cd9a" member_type: NON_VOTER last_known_addr { host: "127.4.159.188" port: 35089 } attrs { promote: true } } }
I20260807 08:48:02.581630 14659 sys_catalog.cc:458] T 00000000000000000000000000000000 P aae8b08fd1ac4d2699d15a396e7ac8f2 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 08:48:02.587483 14677 master.cc:552] Master@127.4.159.188:35089 shutdown complete.
I20260807 08:48:02.587519 14677 master_runner.cc:305] Clearing existing system tablet
I20260807 08:48:02.587620 14677 fs_manager.cc:359] Metadata directory not provided
I20260807 08:48:02.587656 14677 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:48:02.588184 14677 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260807 08:48:02.588598 14841 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:48:02.588713 14677 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:48:02.588766 14677 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 08:48:02.588783 14677 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/master-2/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/master-2/wal
uuid: "383bca56225440c4966e85359729cd9a"
format_stamp: "Formatted at 2026-08-07 08:48:02 on dist-test-slave-6zbq"
server_key: "0607f38541869517f89c1da5d0ea29c9"
server_key_iv: "bcb5d6fdd8c3d8677d0ab1c3fef6b41b"
server_key_version: "encryptionkey@0"
I20260807 08:48:02.589336 14677 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 383bca56225440c4966e85359729cd9a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:48:02.590314 14677 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 383bca56225440c4966e85359729cd9a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260807 08:48:02.590449 14677 log.cc:1196] T 00000000000000000000000000000000 P 383bca56225440c4966e85359729cd9a: Deleting WAL directory at /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260807 08:48:02.590615 14677 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 383bca56225440c4966e85359729cd9a: Deleting consensus metadata
I20260807 08:48:02.590713 14677 master_runner.cc:315] Copying system tablet from 127.4.159.190:37483
I20260807 08:48:02.591214 14677 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 383bca56225440c4966e85359729cd9a: tablet copy: Beginning tablet copy session from remote peer at address 127.4.159.190:37483
I20260807 08:48:02.596580 14371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:02.591482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44878 (local address 127.4.159.190:37483)
0807 08:48:02.591515 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:02.591517 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:02.591618 (+ 101us) server_negotiation.cc:407] Connection header received
0807 08:48:02.591681 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:02.591682 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:02.591690 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:02.591733 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:02.591981 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:02.592417 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:02.593080 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:02.593274 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:02.593325 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:02.594387 (+ 1062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:48:02.594388 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:48:02.594389 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:48:02.594395 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:48:02.595521 (+ 1126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:48:02.595812 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:48:02.595813 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:48:02.595813 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:48:02.595865 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:48:02.596021 (+ 156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:48:02.596022 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:48:02.596022 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:48:02.596232 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:48:02.596290 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 08:48:02.596431 (+ 141us) server_negotiation.cc:299] Negotiation successful
0807 08:48:02.596502 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260807 08:48:02.596618 14850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:02.591454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:37483 (local address 127.0.0.1:44878)
0807 08:48:02.591556 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:02.591566 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:02.591621 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:02.591737 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:02.591737 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:02.591872 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:48:02.591918 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:02.591919 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:02.592483 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:02.592484 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:02.593012 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:02.593013 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:02.593331 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:02.593332 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:02.593394 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:02.594042 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:48:02.594046 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:48:02.594289 (+ 243us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:48:02.595593 (+ 1304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:48:02.595594 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:48:02.595594 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:48:02.595752 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:48:02.595916 (+ 164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:48:02.595917 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:48:02.595917 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:48:02.595972 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:48:02.596299 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:48:02.596299 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:48:02.596342 (+ 43us) client_negotiation.cc:769] Sending connection context
0807 08:48:02.596400 (+ 58us) client_negotiation.cc:240] Negotiation successful
0807 08:48:02.596507 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":83,"thread_start_us":62,"threads_started":1}
I20260807 08:48:02.597105 13687 tablet_copy_service.cc:140] P 82a928c4f0234124b0567479c875bb25: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 383bca56225440c4966e85359729cd9a ({username='oryx', principal='oryx/127.4.159.188@KRBTEST.COM'} at 127.0.0.1:44878)
I20260807 08:48:02.597157 13687 tablet_copy_service.cc:161] P 82a928c4f0234124b0567479c875bb25: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 383bca56225440c4966e85359729cd9a at {username='oryx', principal='oryx/127.4.159.188@KRBTEST.COM'} at 127.0.0.1:44878: session id = 383bca56225440c4966e85359729cd9a-00000000000000000000000000000000
I20260807 08:48:02.597529 13687 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 82a928c4f0234124b0567479c875bb25: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 08:48:02.597929 14677 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260807 08:48:02.598790 14677 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 383bca56225440c4966e85359729cd9a: tablet copy: Starting download of 0 data blocks...
I20260807 08:48:02.598915 14677 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 383bca56225440c4966e85359729cd9a: tablet copy: Starting download of 1 WAL segments...
I20260807 08:48:02.599530 14677 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 383bca56225440c4966e85359729cd9a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 08:48:02.600203 13687 tablet_copy_service.cc:342] P 82a928c4f0234124b0567479c875bb25: Request end of tablet copy session 383bca56225440c4966e85359729cd9a-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.4.159.188@KRBTEST.COM'} at 127.0.0.1:44878
I20260807 08:48:02.600245 13687 tablet_copy_service.cc:433] P 82a928c4f0234124b0567479c875bb25: ending tablet copy session 383bca56225440c4966e85359729cd9a-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 383bca56225440c4966e85359729cd9a
I20260807 08:48:02.602528 14677 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:48:02.603557 14852 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:48:02.603616 14851 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:48:02.603816 14854 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:48:02.603984 14677 server_base.cc:1034] running on GCE node
I20260807 08:48:02.604094 14677 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:48:02.604336 14677 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:48:02.605418 14677 hybrid_clock.cc:648] HybridClock initialized: now 1786092482605409 us; error 23 us; skew 500 ppm
Aug 07 08:48:02 dist-test-slave-6zbq krb5kdc[13584](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092482, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.159.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:48:02.607100 14677 init.cc:377] Logged in from keytab as oryx/127.4.159.188@KRBTEST.COM (short username oryx)
I20260807 08:48:02.607360 14677 webserver.cc:460] Webserver started at http://127.4.159.188:42797/ using document root <none> and password file <none>
I20260807 08:48:02.607470 14677 fs_manager.cc:359] Metadata directory not provided
I20260807 08:48:02.607507 14677 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:48:02.608780 14677 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:48:02.609386 14864 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:48:02.609480 14677 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 08:48:02.609510 14677 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/master-2/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/master-2/wal
uuid: "383bca56225440c4966e85359729cd9a"
format_stamp: "Formatted at 2026-08-07 08:48:02 on dist-test-slave-6zbq"
server_key: "0607f38541869517f89c1da5d0ea29c9"
server_key_iv: "bcb5d6fdd8c3d8677d0ab1c3fef6b41b"
server_key_version: "encryptionkey@0"
I20260807 08:48:02.609552 14677 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:48:02.621323 14677 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:48:02.622658 14677 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:48:02.625595 14677 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.188:35089
I20260807 08:48:02.625682 14942 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.188:35089 every 8 connection(s)
I20260807 08:48:02.625885 14677 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/master-2/data/info.pb
I20260807 08:48:02.626518 14943 sys_catalog.cc:263] Verifying existing consensus state
I20260807 08:48:02.626868 14943 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 383bca56225440c4966e85359729cd9a: Bootstrap starting.
I20260807 08:48:02.629343 14943 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 383bca56225440c4966e85359729cd9a: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 08:48:02.629547 14943 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 383bca56225440c4966e85359729cd9a: Bootstrap complete.
I20260807 08:48:02.629724 14943 raft_consensus.cc:348] T 00000000000000000000000000000000 P 383bca56225440c4966e85359729cd9a [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "82a928c4f0234124b0567479c875bb25" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 37483 } } peers { permanent_uuid: "aae8b08fd1ac4d2699d15a396e7ac8f2" member_type: VOTER last_known_addr { host: "127.4.159.189" port: 37399 } attrs { promote: false } } peers { permanent_uuid: "383bca56225440c4966e85359729cd9a" member_type: NON_VOTER last_known_addr { host: "127.4.159.188" port: 35089 } attrs { promote: true } }
I20260807 08:48:02.629787 14943 raft_consensus.cc:729] T 00000000000000000000000000000000 P 383bca56225440c4966e85359729cd9a [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 383bca56225440c4966e85359729cd9a, State: Initialized, Role: LEARNER
I20260807 08:48:02.629822 14943 consensus_queue.cc:260] T 00000000000000000000000000000000 P 383bca56225440c4966e85359729cd9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "82a928c4f0234124b0567479c875bb25" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 37483 } } peers { permanent_uuid: "aae8b08fd1ac4d2699d15a396e7ac8f2" member_type: VOTER last_known_addr { host: "127.4.159.189" port: 37399 } attrs { promote: false } } peers { permanent_uuid: "383bca56225440c4966e85359729cd9a" member_type: NON_VOTER last_known_addr { host: "127.4.159.188" port: 35089 } attrs { promote: true } }
I20260807 08:48:02.629966 14943 sys_catalog.cc:564] T 00000000000000000000000000000000 P 383bca56225440c4966e85359729cd9a [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:48:02.631317 14969 catalog_manager.cc:1259] Loaded cluster ID: 6e5ed93ced5c45e9b23a567a01055589
I20260807 08:48:02.631363 14969 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 383bca56225440c4966e85359729cd9a: loading cluster ID for follower catalog manager: success
I20260807 08:48:02.632462 14969 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 383bca56225440c4966e85359729cd9a: acquiring CA information for follower catalog manager: success
I20260807 08:48:02.632781 14969 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 383bca56225440c4966e85359729cd9a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 08:48:02.632894 14951 sys_catalog.cc:455] T 00000000000000000000000000000000 P 383bca56225440c4966e85359729cd9a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "82a928c4f0234124b0567479c875bb25" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 37483 } } peers { permanent_uuid: "aae8b08fd1ac4d2699d15a396e7ac8f2" member_type: VOTER last_known_addr { host: "127.4.159.189" port: 37399 } attrs { promote: false } } peers { permanent_uuid: "383bca56225440c4966e85359729cd9a" member_type: NON_VOTER last_known_addr { host: "127.4.159.188" port: 35089 } attrs { promote: true } } }
I20260807 08:48:02.632979 14951 sys_catalog.cc:458] T 00000000000000000000000000000000 P 383bca56225440c4966e85359729cd9a [sys.catalog]: This master's current role is: LEARNER
I20260807 08:48:02.635234 14948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:02.628292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57398 (local address 127.4.159.188:35089)
0807 08:48:02.628422 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:02.628424 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:02.628433 (+ 9us) server_negotiation.cc:407] Connection header received
0807 08:48:02.628453 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:02.628454 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:02.628464 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:02.628542 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:02.629162 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:02.630180 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:02.631137 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:02.631420 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:02.631488 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:02.632953 (+ 1465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:48:02.632954 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:48:02.632954 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:48:02.632961 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:48:02.634206 (+ 1245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:48:02.634551 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:48:02.634552 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:48:02.634553 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:48:02.634608 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:48:02.634776 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:48:02.634776 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:48:02.634776 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:48:02.634875 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:48:02.634956 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 08:48:02.635061 (+ 105us) server_negotiation.cc:299] Negotiation successful
0807 08:48:02.635151 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":71,"threads_started":1}
I20260807 08:48:02.635694 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 13756
I20260807 08:48:02.661521 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.159.129:35375
--local_ip_for_outbound_sockets=127.4.159.129
--tserver_master_addrs=127.4.159.190:37483
--webserver_port=43369
--webserver_interface=127.4.159.129
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/krb5kdc/oryx_127.4.159.129.keytab
--principal=oryx/127.4.159.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:45003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.4.159.190:37483,127.4.159.189:37399
--tserver_master_addrs=127.4.159.190:37483,127.4.159.189:37399,127.4.159.188:35089 with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:48:02.741513 14970 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:48:02.741694 14970 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:48:02.741716 14970 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:48:02.742869 14970 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:48:02.742918 14970 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:48:02.742951 14970 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.159.129
I20260807 08:48:02.744472 14970 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:45003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/krb5kdc/oryx_127.4.159.129.keytab
--principal=oryx/127.4.159.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.159.129:35375
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.159.129
--webserver_port=43369
--tserver_master_addrs=127.4.159.190:37483,127.4.159.189:37399,127.4.159.188:35089
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14970
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.159.129
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:48:02.744674 14970 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:48:02.744889 14970 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:48:02.749507 14976 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:48:02.749553 14970 server_base.cc:1034] running on GCE node
W20260807 08:48:02.749507 14977 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:48:02.749712 14980 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:48:02.749918 14970 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:48:02.750146 14970 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:48:02.751302 14970 hybrid_clock.cc:648] HybridClock initialized: now 1786092482751280 us; error 29 us; skew 500 ppm
Aug 07 08:48:02 dist-test-slave-6zbq krb5kdc[13584](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092482, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.159.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:48:02.753571 14970 init.cc:377] Logged in from keytab as oryx/127.4.159.129@KRBTEST.COM (short username oryx)
I20260807 08:48:02.753949 14970 webserver.cc:460] Webserver started at http://127.4.159.129:43369/ using document root <none> and password file <none>
I20260807 08:48:02.754129 14970 fs_manager.cc:359] Metadata directory not provided
I20260807 08:48:02.754204 14970 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:48:02.755453 14970 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 08:48:02.755947 14993 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:48:02.756095 14970 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 08:48:02.756191 14970 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/ts-0/wal
uuid: "67262c924f4e4493bd8dcab7c92e6aaa"
format_stamp: "Formatted at 2026-08-07 08:48:00 on dist-test-slave-6zbq"
server_key: "efa78947e318008363fee6339f6e564f"
server_key_iv: "7dfd3ed3e1133659e024142d8ff240c2"
server_key_version: "encryptionkey@0"
I20260807 08:48:02.756475 14970 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:48:02.769349 14970 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:48:02.771014 14970 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:48:02.771150 14970 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:48:02.771335 14970 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:48:02.771598 14970 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:48:02.771632 14970 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:48:02.771658 14970 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:48:02.771675 14970 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:48:02.777304 14970 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.129:35375
I20260807 08:48:02.777374 15166 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.129:35375 every 8 connection(s)
I20260807 08:48:02.777659 14970 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/ts-0/data/info.pb
Aug 07 08:48:02 dist-test-slave-6zbq krb5kdc[13584](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092482, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.159.129@KRBTEST.COM for oryx/127.4.159.189@KRBTEST.COM
Aug 07 08:48:02 dist-test-slave-6zbq krb5kdc[13584](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092482, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.159.129@KRBTEST.COM for oryx/127.4.159.188@KRBTEST.COM
Aug 07 08:48:02 dist-test-slave-6zbq krb5kdc[13584](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092482, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.159.129@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM
I20260807 08:48:02.786388 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 14970
I20260807 08:48:02.786547 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 13954
I20260807 08:48:02.798461 14371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:02.778640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.129:33951 (local address 127.4.159.190:37483)
0807 08:48:02.778686 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:02.778688 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:02.778845 (+ 157us) server_negotiation.cc:407] Connection header received
0807 08:48:02.779689 (+ 844us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:02.779691 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:02.779703 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:02.779755 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:02.780401 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:02.781161 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:02.782545 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:02.782764 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:02.782826 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:02.785493 (+ 2667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:48:02.785495 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:48:02.785495 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:48:02.785503 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:48:02.793935 (+ 8432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:48:02.797623 (+ 3688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:48:02.797624 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:48:02.797624 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:48:02.797682 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:48:02.797866 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:48:02.797867 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:48:02.797868 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:48:02.797989 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:48:02.798027 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0807 08:48:02.798258 (+ 231us) server_negotiation.cc:299] Negotiation successful
0807 08:48:02.798364 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 08:48:02.798470 14674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:02.778455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.129:47841 (local address 127.4.159.189:37399)
0807 08:48:02.778500 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:02.778502 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:02.778774 (+ 272us) server_negotiation.cc:407] Connection header received
0807 08:48:02.779812 (+ 1038us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:02.779813 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:02.779825 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:02.779876 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:02.780392 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:02.780868 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:02.782152 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:02.782446 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:02.782497 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:02.784551 (+ 2054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:48:02.784552 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:48:02.784552 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:48:02.784559 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:48:02.786245 (+ 1686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:48:02.797350 (+ 11105us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:48:02.797352 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:48:02.797352 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:48:02.797419 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:48:02.797691 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:48:02.797692 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:48:02.797692 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:48:02.797853 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:48:02.797897 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0807 08:48:02.798258 (+ 361us) server_negotiation.cc:299] Negotiation successful
0807 08:48:02.798374 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 08:48:02.799036 15173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:02.778577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:37483 (local address 127.4.159.129:33951)
0807 08:48:02.778765 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:02.778774 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:02.779529 (+ 755us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:02.779771 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:02.779772 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:02.780153 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:48:02.780283 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:02.780290 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:02.781239 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:02.781240 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:02.782072 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:02.782073 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:02.782834 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:02.782835 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:02.782898 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:02.783205 (+ 307us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:48:02.783209 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:48:02.785383 (+ 2174us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:48:02.797348 (+ 11965us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:48:02.797348 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:48:02.797349 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:48:02.797543 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:48:02.797752 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:48:02.797752 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:48:02.797753 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:48:02.797802 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:48:02.798068 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:48:02.798069 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:48:02.798104 (+ 35us) client_negotiation.cc:769] Sending connection context
0807 08:48:02.798583 (+ 479us) client_negotiation.cc:240] Negotiation successful
0807 08:48:02.798856 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"mutex_wait_us":49,"thread_start_us":67,"threads_started":1}
I20260807 08:48:02.799433 15167 heartbeater.cc:347] Connected to a master server at 127.4.159.190:37483
I20260807 08:48:02.799541 15167 heartbeater.cc:464] Registering TS with master...
I20260807 08:48:02.799777 15167 heartbeater.cc:511] Master 127.4.159.190:37483 requested a full tablet report, sending...
I20260807 08:48:02.800104 14948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:02.778381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.129:54303 (local address 127.4.159.188:35089)
0807 08:48:02.778434 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:02.778437 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:02.778678 (+ 241us) server_negotiation.cc:407] Connection header received
0807 08:48:02.779623 (+ 945us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:02.779625 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:02.779638 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:02.779687 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:02.780378 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:02.781158 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:02.781991 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:02.782240 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:02.782308 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:02.784971 (+ 2663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:48:02.784972 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:48:02.784972 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:48:02.784980 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:48:02.786377 (+ 1397us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:48:02.799443 (+ 13066us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:48:02.799445 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:48:02.799446 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:48:02.799505 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:48:02.799668 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:48:02.799668 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:48:02.799669 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:48:02.799774 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:48:02.799813 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0807 08:48:02.799952 (+ 139us) server_negotiation.cc:299] Negotiation successful
0807 08:48:02.800028 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 08:48:02.800252 13654 ts_manager.cc:194] Re-registered known tserver with Master: 67262c924f4e4493bd8dcab7c92e6aaa (127.4.159.129:35375)
I20260807 08:48:02.800421 15172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:02.778270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.188:35089 (local address 127.4.159.129:54303)
0807 08:48:02.778467 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:02.778507 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:02.779505 (+ 998us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:02.779758 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:02.779762 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:02.780100 (+ 338us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:48:02.780282 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:02.780286 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:02.781270 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:02.781271 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:02.781920 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:02.781922 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:02.782314 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:02.782314 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:02.782459 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:02.782982 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:48:02.782991 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:48:02.784881 (+ 1890us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:48:02.786470 (+ 1589us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:48:02.786478 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:48:02.786481 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:48:02.799354 (+ 12873us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:48:02.799561 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:48:02.799562 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:48:02.799562 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:48:02.799607 (+ 45us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:48:02.799836 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:48:02.799837 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:48:02.799870 (+ 33us) client_negotiation.cc:769] Sending connection context
0807 08:48:02.800193 (+ 323us) client_negotiation.cc:240] Negotiation successful
0807 08:48:02.800304 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":104,"threads_started":1}
I20260807 08:48:02.800649 13654 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.4.159.129@KRBTEST.COM'} at 127.4.159.129:33951
I20260807 08:48:02.800927 15175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:02.778524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.189:37399 (local address 127.4.159.129:47841)
0807 08:48:02.778714 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:02.778723 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:02.779516 (+ 793us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:02.779894 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:02.779895 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:02.780153 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:48:02.780283 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:02.780290 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:02.780966 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:02.780967 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:02.782067 (+ 1100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:02.782070 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:02.782507 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:02.782508 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:02.782563 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:02.783079 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:48:02.783083 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:48:02.784464 (+ 1381us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:48:02.797051 (+ 12587us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:48:02.797053 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:48:02.797053 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:48:02.797261 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:48:02.797492 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:48:02.797493 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:48:02.797493 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:48:02.797617 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:48:02.797943 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:48:02.797950 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:48:02.798104 (+ 154us) client_negotiation.cc:769] Sending connection context
0807 08:48:02.798627 (+ 523us) client_negotiation.cc:240] Negotiation successful
0807 08:48:02.800848 (+ 2221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"mutex_wait_us":2130,"thread_start_us":82,"threads_started":1}
I20260807 08:48:02.800983 15168 heartbeater.cc:347] Connected to a master server at 127.4.159.188:35089
I20260807 08:48:02.801030 15168 heartbeater.cc:464] Registering TS with master...
I20260807 08:48:02.801112 15168 heartbeater.cc:511] Master 127.4.159.188:35089 requested a full tablet report, sending...
I20260807 08:48:02.801221 15170 heartbeater.cc:347] Connected to a master server at 127.4.159.189:37399
I20260807 08:48:02.801254 15170 heartbeater.cc:464] Registering TS with master...
I20260807 08:48:02.801312 15170 heartbeater.cc:511] Master 127.4.159.189:37399 requested a full tablet report, sending...
I20260807 08:48:02.801590 14599 ts_manager.cc:194] Registered new tserver with Master: 67262c924f4e4493bd8dcab7c92e6aaa (127.4.159.129:35375)
I20260807 08:48:02.802147 14885 ts_manager.cc:194] Registered new tserver with Master: 67262c924f4e4493bd8dcab7c92e6aaa (127.4.159.129:35375)
I20260807 08:48:02.807240 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.159.130:39079
--local_ip_for_outbound_sockets=127.4.159.130
--tserver_master_addrs=127.4.159.190:37483
--webserver_port=35819
--webserver_interface=127.4.159.130
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/krb5kdc/oryx_127.4.159.130.keytab
--principal=oryx/127.4.159.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:45003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.4.159.190:37483,127.4.159.189:37399
--tserver_master_addrs=127.4.159.190:37483,127.4.159.189:37399,127.4.159.188:35089 with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:48:02.887753 15176 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:48:02.887929 15176 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:48:02.887960 15176 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:48:02.889053 15176 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:48:02.889091 15176 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:48:02.889125 15176 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.159.130
I20260807 08:48:02.890583 15176 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:45003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/krb5kdc/oryx_127.4.159.130.keytab
--principal=oryx/127.4.159.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.159.130:39079
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.159.130
--webserver_port=35819
--tserver_master_addrs=127.4.159.190:37483,127.4.159.189:37399,127.4.159.188:35089
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15176
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.159.130
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:48:02.890811 15176 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:48:02.890997 15176 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:48:02.895474 15187 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:48:02.895551 15176 server_base.cc:1034] running on GCE node
W20260807 08:48:02.895493 15186 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:48:02.895493 15194 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:48:02.895843 15176 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:48:02.896062 15176 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:48:02.897193 15176 hybrid_clock.cc:648] HybridClock initialized: now 1786092482897175 us; error 31 us; skew 500 ppm
Aug 07 08:48:02 dist-test-slave-6zbq krb5kdc[13584](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092482, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.159.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:48:02.899466 15176 init.cc:377] Logged in from keytab as oryx/127.4.159.130@KRBTEST.COM (short username oryx)
I20260807 08:48:02.899837 15176 webserver.cc:460] Webserver started at http://127.4.159.130:35819/ using document root <none> and password file <none>
I20260807 08:48:02.900022 15176 fs_manager.cc:359] Metadata directory not provided
I20260807 08:48:02.900080 15176 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:48:02.901284 15176 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:48:02.901790 15207 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:48:02.901942 15176 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 08:48:02.902040 15176 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/ts-1/wal
uuid: "bed5f8e9f0ba46ea97101418a4f0affe"
format_stamp: "Formatted at 2026-08-07 08:48:01 on dist-test-slave-6zbq"
server_key: "3e9967887dc223e89823909cb109304b"
server_key_iv: "e65e9138feb753c604ed8b3c120937ba"
server_key_version: "encryptionkey@0"
I20260807 08:48:02.902334 15176 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:48:02.924748 15176 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:48:02.926265 15176 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:48:02.926409 15176 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:48:02.926591 15176 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:48:02.926846 15176 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:48:02.926878 15176 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:48:02.926903 15176 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:48:02.926920 15176 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:48:02.932201 15176 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.130:39079
I20260807 08:48:02.932260 15385 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.130:39079 every 8 connection(s)
I20260807 08:48:02.932565 15176 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/ts-1/data/info.pb
Aug 07 08:48:02 dist-test-slave-6zbq krb5kdc[13584](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092482, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.159.130@KRBTEST.COM for oryx/127.4.159.188@KRBTEST.COM
Aug 07 08:48:02 dist-test-slave-6zbq krb5kdc[13584](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092482, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.159.130@KRBTEST.COM for oryx/127.4.159.189@KRBTEST.COM
Aug 07 08:48:02 dist-test-slave-6zbq krb5kdc[13584](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092482, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.159.130@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM
I20260807 08:48:02.942399 14674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:02.933196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.130:58691 (local address 127.4.159.189:37399)
0807 08:48:02.933241 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:02.933242 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:02.933428 (+ 186us) server_negotiation.cc:407] Connection header received
0807 08:48:02.933967 (+ 539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:02.933968 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:02.933979 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:02.934010 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:02.934888 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:02.935640 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:02.936768 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:02.936953 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:02.937005 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:02.939588 (+ 2583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:48:02.939590 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:48:02.939590 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:48:02.939597 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:48:02.941034 (+ 1437us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:48:02.941397 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:48:02.941398 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:48:02.941399 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:48:02.941461 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:48:02.941748 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:48:02.941749 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:48:02.941749 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:48:02.941866 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:48:02.941922 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 08:48:02.942226 (+ 304us) server_negotiation.cc:299] Negotiation successful
0807 08:48:02.942292 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 08:48:02.942394 14948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:02.933572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.130:39763 (local address 127.4.159.188:35089)
0807 08:48:02.933621 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:02.933623 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:02.933632 (+ 9us) server_negotiation.cc:407] Connection header received
0807 08:48:02.933966 (+ 334us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:02.933967 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:02.933979 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:02.934009 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:02.935742 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:02.936203 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:02.937087 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:02.937373 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:02.937433 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:02.939227 (+ 1794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:48:02.939228 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:48:02.939228 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:48:02.939234 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:48:02.940868 (+ 1634us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:48:02.941428 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:48:02.941429 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:48:02.941430 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:48:02.941483 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:48:02.941715 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:48:02.941716 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:48:02.941716 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:48:02.941843 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:48:02.941913 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 08:48:02.942192 (+ 279us) server_negotiation.cc:299] Negotiation successful
0807 08:48:02.942271 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 08:48:02.942600 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 15176
I20260807 08:48:02.942759 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 14177
I20260807 08:48:02.942826 14371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:02.933134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.130:60807 (local address 127.4.159.190:37483)
0807 08:48:02.933177 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:02.933179 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:02.933455 (+ 276us) server_negotiation.cc:407] Connection header received
0807 08:48:02.933948 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:02.933949 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:02.933963 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:02.934014 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:02.934863 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:02.935647 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:02.937503 (+ 1856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:02.937790 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:02.937864 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:02.940114 (+ 2250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:48:02.940116 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:48:02.940116 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:48:02.940124 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:48:02.941658 (+ 1534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:48:02.941986 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:48:02.941988 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:48:02.941988 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:48:02.942053 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:48:02.942258 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:48:02.942259 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:48:02.942260 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:48:02.942413 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:48:02.942459 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0807 08:48:02.942662 (+ 203us) server_negotiation.cc:299] Negotiation successful
0807 08:48:02.942739 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260807 08:48:02.942806 15394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:02.933151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.189:37399 (local address 127.4.159.130:58691)
0807 08:48:02.933321 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:02.933353 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:02.933849 (+ 496us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:02.934084 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:02.934089 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:02.934566 (+ 477us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:48:02.934770 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:02.934776 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:02.935736 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:02.935736 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:02.936678 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:02.936680 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:02.937036 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:02.937037 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:02.937171 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:02.937719 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:48:02.937728 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:48:02.939517 (+ 1789us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:48:02.941113 (+ 1596us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:48:02.941114 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:48:02.941114 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:48:02.941336 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:48:02.941526 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:48:02.941527 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:48:02.941527 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:48:02.941685 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:48:02.941950 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:48:02.941951 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:48:02.942076 (+ 125us) client_negotiation.cc:769] Sending connection context
0807 08:48:02.942540 (+ 464us) client_negotiation.cc:240] Negotiation successful
0807 08:48:02.942638 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"mutex_wait_us":4,"thread_start_us":85,"threads_started":1}
I20260807 08:48:02.946066 15395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:02.933150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:37483 (local address 127.4.159.130:60807)
0807 08:48:02.933391 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:02.933401 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:02.933857 (+ 456us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:02.934370 (+ 513us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:02.934371 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:02.934580 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:48:02.934769 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:02.934774 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:02.936773 (+ 1999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:02.936774 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:02.937396 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:02.937399 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:02.937874 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:02.937875 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:02.937938 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:02.938333 (+ 395us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:48:02.938359 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:48:02.939985 (+ 1626us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:48:02.941739 (+ 1754us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:48:02.941740 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:48:02.941740 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:48:02.941895 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:48:02.942108 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:48:02.942109 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:48:02.942109 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:48:02.942171 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:48:02.942512 (+ 341us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:48:02.942513 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:48:02.942551 (+ 38us) client_negotiation.cc:769] Sending connection context
0807 08:48:02.942894 (+ 343us) client_negotiation.cc:240] Negotiation successful
0807 08:48:02.942982 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"mutex_wait_us":5,"spinlock_wait_cycles":6528,"thread_start_us":87,"threads_started":1}
I20260807 08:48:02.946650 15392 heartbeater.cc:347] Connected to a master server at 127.4.159.189:37399
I20260807 08:48:02.946733 15392 heartbeater.cc:464] Registering TS with master...
I20260807 08:48:02.946889 15392 heartbeater.cc:511] Master 127.4.159.189:37399 requested a full tablet report, sending...
I20260807 08:48:02.947325 14599 ts_manager.cc:194] Registered new tserver with Master: bed5f8e9f0ba46ea97101418a4f0affe (127.4.159.130:39079)
I20260807 08:48:02.947353 15396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:02.933420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.188:35089 (local address 127.4.159.130:39763)
0807 08:48:02.933520 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:02.933530 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:02.933862 (+ 332us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:02.934095 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:02.934096 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:02.934590 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:48:02.934770 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:02.934776 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:02.936278 (+ 1502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:02.936279 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:02.937026 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:02.937028 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:02.937442 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:02.937443 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:02.937504 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:02.937812 (+ 308us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:48:02.937818 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:48:02.939135 (+ 1317us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:48:02.940946 (+ 1811us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:48:02.940971 (+ 25us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:48:02.940973 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:48:02.941325 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:48:02.941556 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:48:02.941557 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:48:02.941557 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:48:02.941651 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:48:02.941912 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:48:02.941918 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:48:02.942064 (+ 146us) client_negotiation.cc:769] Sending connection context
0807 08:48:02.942167 (+ 103us) client_negotiation.cc:240] Negotiation successful
0807 08:48:02.942320 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":86,"mutex_wait_us":18,"thread_start_us":61,"threads_started":1}
I20260807 08:48:02.947651 15390 heartbeater.cc:347] Connected to a master server at 127.4.159.188:35089
I20260807 08:48:02.947696 15390 heartbeater.cc:464] Registering TS with master...
I20260807 08:48:02.947752 15390 heartbeater.cc:511] Master 127.4.159.188:35089 requested a full tablet report, sending...
I20260807 08:48:02.948014 14885 ts_manager.cc:194] Registered new tserver with Master: bed5f8e9f0ba46ea97101418a4f0affe (127.4.159.130:39079)
I20260807 08:48:02.949958 15389 heartbeater.cc:347] Connected to a master server at 127.4.159.190:37483
I20260807 08:48:02.950013 15389 heartbeater.cc:464] Registering TS with master...
I20260807 08:48:02.950076 15389 heartbeater.cc:511] Master 127.4.159.190:37483 requested a full tablet report, sending...
I20260807 08:48:02.950114 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.159.131:43209
--local_ip_for_outbound_sockets=127.4.159.131
--tserver_master_addrs=127.4.159.190:37483
--webserver_port=40137
--webserver_interface=127.4.159.131
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/krb5kdc/oryx_127.4.159.131.keytab
--principal=oryx/127.4.159.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:45003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.4.159.190:37483,127.4.159.189:37399
--tserver_master_addrs=127.4.159.190:37483,127.4.159.189:37399,127.4.159.188:35089 with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 08:48:02.950289 13654 ts_manager.cc:194] Re-registered known tserver with Master: bed5f8e9f0ba46ea97101418a4f0affe (127.4.159.130:39079)
I20260807 08:48:02.950655 13654 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.4.159.130@KRBTEST.COM'} at 127.4.159.130:60807
W20260807 08:48:03.029422 15397 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:48:03.029598 15397 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:48:03.029620 15397 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:48:03.030819 15397 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:48:03.030858 15397 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:48:03.030887 15397 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.159.131
I20260807 08:48:03.032331 15397 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:45003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/krb5kdc/oryx_127.4.159.131.keytab
--principal=oryx/127.4.159.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.159.131:43209
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.159.131
--webserver_port=40137
--tserver_master_addrs=127.4.159.190:37483,127.4.159.189:37399,127.4.159.188:35089
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15397
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.159.131
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:48:03.032528 15397 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:48:03.032747 15397 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:48:03.037305 15408 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:48:03.037384 15397 server_base.cc:1034] running on GCE node
W20260807 08:48:03.037328 15406 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:48:03.037313 15412 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:48:03.037669 15397 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:48:03.037891 15397 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:48:03.039021 15397 hybrid_clock.cc:648] HybridClock initialized: now 1786092483039007 us; error 29 us; skew 500 ppm
Aug 07 08:48:03 dist-test-slave-6zbq krb5kdc[13584](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092483, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.159.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:48:03.041267 15397 init.cc:377] Logged in from keytab as oryx/127.4.159.131@KRBTEST.COM (short username oryx)
I20260807 08:48:03.041618 15397 webserver.cc:460] Webserver started at http://127.4.159.131:40137/ using document root <none> and password file <none>
I20260807 08:48:03.041819 15397 fs_manager.cc:359] Metadata directory not provided
I20260807 08:48:03.041877 15397 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:48:03.043169 15397 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:48:03.043726 15420 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:48:03.043879 15397 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 08:48:03.043978 15397 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/ts-2/wal
uuid: "48c69edfbb324b37ace3f551898f2220"
format_stamp: "Formatted at 2026-08-07 08:48:02 on dist-test-slave-6zbq"
server_key: "c17f15525e12913055c2425a4b38b0a5"
server_key_iv: "e0a0cf1c150d54d28a6fab46d4249a36"
server_key_version: "encryptionkey@0"
I20260807 08:48:03.044302 15397 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:48:03.052985 15397 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:48:03.054900 15397 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:48:03.054999 15397 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:48:03.055157 15397 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:48:03.055377 15397 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:48:03.055406 15397 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:48:03.055429 15397 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:48:03.055446 15397 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:48:03.060902 15397 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.131:43209
I20260807 08:48:03.060963 15580 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.131:43209 every 8 connection(s)
I20260807 08:48:03.061249 15397 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/ts-2/data/info.pb
I20260807 08:48:03.064882 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 15397
Aug 07 08:48:03 dist-test-slave-6zbq krb5kdc[13584](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092483, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.159.131@KRBTEST.COM for oryx/127.4.159.188@KRBTEST.COM
Aug 07 08:48:03 dist-test-slave-6zbq krb5kdc[13584](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092483, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.159.131@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM
Aug 07 08:48:03 dist-test-slave-6zbq krb5kdc[13584](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092483, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.159.131@KRBTEST.COM for oryx/127.4.159.189@KRBTEST.COM
I20260807 08:48:03.071413 15590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:03.061856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.188:35089 (local address 127.4.159.131:45171)
0807 08:48:03.062021 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:03.062059 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:03.063013 (+ 954us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:03.063243 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:03.063245 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:03.063609 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:48:03.063790 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:03.063795 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:03.064746 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:03.064746 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:03.065412 (+ 666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:03.065414 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:03.065836 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:03.065837 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:03.065942 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:03.066523 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:48:03.066529 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:48:03.068163 (+ 1634us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:48:03.069871 (+ 1708us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:48:03.069877 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:48:03.069879 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:48:03.070139 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:48:03.070366 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:48:03.070367 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:48:03.070367 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:48:03.070481 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:48:03.070795 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:48:03.070802 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:48:03.070949 (+ 147us) client_negotiation.cc:769] Sending connection context
0807 08:48:03.071056 (+ 107us) client_negotiation.cc:240] Negotiation successful
0807 08:48:03.071209 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":97,"threads_started":1}
I20260807 08:48:03.071709 14948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:03.061893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.131:45171 (local address 127.4.159.188:35089)
0807 08:48:03.061947 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:03.061949 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:03.062174 (+ 225us) server_negotiation.cc:407] Connection header received
0807 08:48:03.063116 (+ 942us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:03.063117 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:03.063130 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:03.063180 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:03.063876 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:03.064662 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:03.065491 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:03.065753 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:03.065806 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:03.068273 (+ 2467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:48:03.068274 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:48:03.068274 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:48:03.068281 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:48:03.069799 (+ 1518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:48:03.070211 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:48:03.070212 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:48:03.070213 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:48:03.070276 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:48:03.070553 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:48:03.070554 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:48:03.070554 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:48:03.070693 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:48:03.070766 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 08:48:03.071548 (+ 782us) server_negotiation.cc:299] Negotiation successful
0807 08:48:03.071624 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260807 08:48:03.072499 15584 heartbeater.cc:347] Connected to a master server at 127.4.159.188:35089
I20260807 08:48:03.072587 15584 heartbeater.cc:464] Registering TS with master...
I20260807 08:48:03.072749 15584 heartbeater.cc:511] Master 127.4.159.188:35089 requested a full tablet report, sending...
I20260807 08:48:03.073019 15592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:03.061855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.189:37399 (local address 127.4.159.131:38551)
0807 08:48:03.062256 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:03.062267 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:03.063016 (+ 749us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:03.063242 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:03.063245 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:03.063638 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:48:03.063791 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:03.063797 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:03.065839 (+ 2042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:03.065840 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:03.066569 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:03.066572 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:03.067084 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:03.067085 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:03.067146 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:03.067490 (+ 344us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:48:03.067494 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:48:03.069467 (+ 1973us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:48:03.072051 (+ 2584us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:48:03.072052 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:48:03.072052 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:48:03.072255 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:48:03.072447 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:48:03.072447 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:48:03.072447 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:48:03.072497 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:48:03.072735 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:48:03.072735 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:48:03.072775 (+ 40us) client_negotiation.cc:769] Sending connection context
0807 08:48:03.072831 (+ 56us) client_negotiation.cc:240] Negotiation successful
0807 08:48:03.072916 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":367,"mutex_wait_us":30,"thread_start_us":84,"threads_started":1}
I20260807 08:48:03.073129 14885 ts_manager.cc:194] Registered new tserver with Master: 48c69edfbb324b37ace3f551898f2220 (127.4.159.131:43209)
I20260807 08:48:03.073370 14674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:03.061848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.131:38551 (local address 127.4.159.189:37399)
0807 08:48:03.061956 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:03.061958 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:03.062315 (+ 357us) server_negotiation.cc:407] Connection header received
0807 08:48:03.063124 (+ 809us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:03.063126 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:03.063138 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:03.063167 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:03.063903 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:03.064651 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:03.066664 (+ 2013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:03.066962 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:03.067006 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:03.069566 (+ 2560us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:48:03.069568 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:48:03.069568 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:48:03.069575 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:48:03.071977 (+ 2402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:48:03.072332 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:48:03.072333 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:48:03.072334 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:48:03.072390 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:48:03.072551 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:48:03.072552 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:48:03.072552 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:48:03.072662 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:48:03.072710 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0807 08:48:03.073188 (+ 478us) server_negotiation.cc:299] Negotiation successful
0807 08:48:03.073277 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 08:48:03.073627 15586 heartbeater.cc:347] Connected to a master server at 127.4.159.189:37399
I20260807 08:48:03.073670 15586 heartbeater.cc:464] Registering TS with master...
I20260807 08:48:03.073725 15586 heartbeater.cc:511] Master 127.4.159.189:37399 requested a full tablet report, sending...
I20260807 08:48:03.074007 14599 ts_manager.cc:194] Registered new tserver with Master: 48c69edfbb324b37ace3f551898f2220 (127.4.159.131:43209)
I20260807 08:48:03.074927 14527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:03.070757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.188:35089 (local address 127.0.0.1:57406)
0807 08:48:03.070835 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:03.070844 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:03.070877 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:03.071938 (+ 1061us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:03.071939 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:03.071941 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:48:03.071976 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:03.071977 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:03.072783 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:03.072785 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:03.073836 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:03.073839 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:03.074680 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:03.074681 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:03.074756 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:03.074759 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 08:48:03.074809 (+ 50us) client_negotiation.cc:240] Negotiation successful
0807 08:48:03.074853 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62}
I20260807 08:48:03.075171 14948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:03.070943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57406 (local address 127.4.159.188:35089)
0807 08:48:03.071786 (+ 843us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:03.071789 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:03.071796 (+ 7us) server_negotiation.cc:407] Connection header received
0807 08:48:03.071814 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:03.071815 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:03.071835 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:03.071896 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:48:03.072045 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:03.072690 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:03.073920 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:03.074510 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:03.074574 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:03.074643 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0807 08:48:03.075076 (+ 433us) server_negotiation.cc:299] Negotiation successful
0807 08:48:03.075114 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":829,"thread_start_us":49,"threads_started":1}
I20260807 08:48:03.075439 14900 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 383bca56225440c4966e85359729cd9a [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260807 08:48:03.075666 14951 sys_catalog.cc:455] T 00000000000000000000000000000000 P 383bca56225440c4966e85359729cd9a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 82a928c4f0234124b0567479c875bb25. Latest consensus state: current_term: 1 leader_uuid: "82a928c4f0234124b0567479c875bb25" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "82a928c4f0234124b0567479c875bb25" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 37483 } } peers { permanent_uuid: "aae8b08fd1ac4d2699d15a396e7ac8f2" member_type: VOTER last_known_addr { host: "127.4.159.189" port: 37399 } attrs { promote: false } } peers { permanent_uuid: "383bca56225440c4966e85359729cd9a" member_type: NON_VOTER last_known_addr { host: "127.4.159.188" port: 35089 } attrs { promote: true } } }
I20260807 08:48:03.075752 14951 sys_catalog.cc:458] T 00000000000000000000000000000000 P 383bca56225440c4966e85359729cd9a [sys.catalog]: This master's current role is: LEARNER
I20260807 08:48:03.076367 15589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:03.062129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:37483 (local address 127.4.159.131:57569)
0807 08:48:03.062393 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:03.062400 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:03.063023 (+ 623us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:03.063315 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:03.063316 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:03.063646 (+ 330us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:48:03.063791 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:03.063797 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:03.064484 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:03.064485 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:03.065481 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:03.065484 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:03.065966 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:03.065967 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:03.066027 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:03.066514 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:48:03.066523 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:48:03.069191 (+ 2668us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:48:03.075005 (+ 5814us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:48:03.075005 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:48:03.075006 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:48:03.075216 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:48:03.075737 (+ 521us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:48:03.075738 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:48:03.075738 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:48:03.075800 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:48:03.076090 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:48:03.076092 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:48:03.076132 (+ 40us) client_negotiation.cc:769] Sending connection context
0807 08:48:03.076209 (+ 77us) client_negotiation.cc:240] Negotiation successful
0807 08:48:03.076293 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"mutex_wait_us":2,"thread_start_us":69,"threads_started":1}
I20260807 08:48:03.076676 14371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:03.062198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.131:57569 (local address 127.4.159.190:37483)
0807 08:48:03.062258 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:03.062260 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:03.062434 (+ 174us) server_negotiation.cc:407] Connection header received
0807 08:48:03.063238 (+ 804us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:03.063239 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:03.063250 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:03.063300 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:03.063899 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:03.064382 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:03.065588 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:03.065853 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:03.065895 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:03.069262 (+ 3367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:48:03.069264 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:48:03.069264 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:48:03.069271 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:48:03.074911 (+ 5640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:48:03.075599 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:48:03.075600 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:48:03.075600 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:48:03.075661 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:48:03.075898 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:48:03.075899 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:48:03.075900 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:48:03.076025 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:48:03.076096 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 08:48:03.076529 (+ 433us) server_negotiation.cc:299] Negotiation successful
0807 08:48:03.076604 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260807 08:48:03.077189 15583 heartbeater.cc:347] Connected to a master server at 127.4.159.190:37483
I20260807 08:48:03.077234 15583 heartbeater.cc:464] Registering TS with master...
I20260807 08:48:03.077293 15583 heartbeater.cc:511] Master 127.4.159.190:37483 requested a full tablet report, sending...
I20260807 08:48:03.077499 13654 ts_manager.cc:194] Re-registered known tserver with Master: 48c69edfbb324b37ace3f551898f2220 (127.4.159.131:43209)
I20260807 08:48:03.077770 15603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:03.066327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44882 (local address 127.4.159.190:37483)
0807 08:48:03.066679 (+ 352us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:03.066681 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:03.070032 (+ 3351us) server_negotiation.cc:407] Connection header received
0807 08:48:03.070411 (+ 379us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:03.070412 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:03.070425 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:03.070469 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:03.071341 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:03.072074 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:03.073177 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:03.073467 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:03.073508 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:03.075024 (+ 1516us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:48:03.075025 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:48:03.075026 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:48:03.075032 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:48:03.076559 (+ 1527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:48:03.076910 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:48:03.076911 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:48:03.076911 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:48:03.076969 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:48:03.077170 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:48:03.077171 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:48:03.077171 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:48:03.077292 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:48:03.077364 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 08:48:03.077544 (+ 180us) server_negotiation.cc:299] Negotiation successful
0807 08:48:03.077623 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":99,"threads_started":1}
I20260807 08:48:03.077836 13654 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.4.159.131@KRBTEST.COM'} at 127.4.159.131:57569
I20260807 08:48:03.079350 15602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:03.066362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49264 (local address 127.4.159.189:37399)
0807 08:48:03.066734 (+ 372us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:03.066736 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:03.066820 (+ 84us) server_negotiation.cc:407] Connection header received
0807 08:48:03.066873 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:03.066874 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:03.066893 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:03.066948 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:03.067999 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:03.068755 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:03.069750 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:03.070028 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:03.070088 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:03.071434 (+ 1346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:48:03.071435 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:48:03.071435 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:48:03.071443 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:48:03.078079 (+ 6636us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:48:03.078518 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:48:03.078519 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:48:03.078519 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:48:03.078582 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:48:03.078799 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:48:03.078801 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:48:03.078801 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:48:03.078937 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:48:03.079036 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0807 08:48:03.079175 (+ 139us) server_negotiation.cc:299] Negotiation successful
0807 08:48:03.079259 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":58,"threads_started":1}
I20260807 08:48:03.079679 13654 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44882:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 08:48:03.081471 13654 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:48:03.082162 14837 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82a928c4f0234124b0567479c875bb25 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "82a928c4f0234124b0567479c875bb25" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "82a928c4f0234124b0567479c875bb25" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 37483 } } peers { permanent_uuid: "aae8b08fd1ac4d2699d15a396e7ac8f2" member_type: VOTER last_known_addr { host: "127.4.159.189" port: 37399 } attrs { promote: false } } peers { permanent_uuid: "383bca56225440c4966e85359729cd9a" member_type: NON_VOTER last_known_addr { host: "127.4.159.188" port: 35089 } attrs { promote: true } } }
I20260807 08:48:03.082250 14837 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82a928c4f0234124b0567479c875bb25 [sys.catalog]: This master's current role is: LEADER
I20260807 08:48:03.082979 15608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:03.067818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57404 (local address 127.4.159.188:35089)
0807 08:48:03.069925 (+ 2107us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:03.069927 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:03.070129 (+ 202us) server_negotiation.cc:407] Connection header received
0807 08:48:03.070270 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:03.070270 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:03.070284 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:03.070319 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:03.075968 (+ 5649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:03.076661 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:03.078204 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:03.078523 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:03.078560 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:03.080368 (+ 1808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:48:03.080370 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:48:03.080370 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:48:03.080376 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:48:03.081750 (+ 1374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:48:03.082117 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:48:03.082119 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:48:03.082119 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:48:03.082180 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:48:03.082413 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:48:03.082414 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:48:03.082415 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:48:03.082532 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:48:03.082599 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 08:48:03.082813 (+ 214us) server_negotiation.cc:299] Negotiation successful
0807 08:48:03.082896 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2088,"thread_start_us":62,"threads_started":1}
I20260807 08:48:03.083526 15619 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 82a928c4f0234124b0567479c875bb25: attempting to promote NON_VOTER 383bca56225440c4966e85359729cd9a to VOTER
I20260807 08:48:03.083647 15619 consensus_queue.cc:237] T 00000000000000000000000000000000 P 82a928c4f0234124b0567479c875bb25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "82a928c4f0234124b0567479c875bb25" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 37483 } } peers { permanent_uuid: "aae8b08fd1ac4d2699d15a396e7ac8f2" member_type: VOTER last_known_addr { host: "127.4.159.189" port: 37399 } attrs { promote: false } } peers { permanent_uuid: "383bca56225440c4966e85359729cd9a" member_type: VOTER last_known_addr { host: "127.4.159.188" port: 35089 } attrs { promote: false } }
I20260807 08:48:03.084038 14900 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 383bca56225440c4966e85359729cd9a [term 1 LEARNER]: Refusing update from remote peer 82a928c4f0234124b0567479c875bb25: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260807 08:48:03.084183 14616 raft_consensus.cc:1261] T 00000000000000000000000000000000 P aae8b08fd1ac4d2699d15a396e7ac8f2 [term 1 FOLLOWER]: Refusing update from remote peer 82a928c4f0234124b0567479c875bb25: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260807 08:48:03.084268 15614 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 82a928c4f0234124b0567479c875bb25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "383bca56225440c4966e85359729cd9a" member_type: VOTER last_known_addr { host: "127.4.159.188" port: 35089 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260807 08:48:03.084352 14837 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 82a928c4f0234124b0567479c875bb25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aae8b08fd1ac4d2699d15a396e7ac8f2" member_type: VOTER last_known_addr { host: "127.4.159.189" port: 37399 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260807 08:48:03.084926 14837 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 82a928c4f0234124b0567479c875bb25 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 383bca56225440c4966e85359729cd9a (127.4.159.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "82a928c4f0234124b0567479c875bb25" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 37483 } } peers { permanent_uuid: "aae8b08fd1ac4d2699d15a396e7ac8f2" member_type: VOTER last_known_addr { host: "127.4.159.189" port: 37399 } attrs { promote: false } } peers { permanent_uuid: "383bca56225440c4966e85359729cd9a" member_type: VOTER last_known_addr { host: "127.4.159.188" port: 35089 } attrs { promote: false } } }
I20260807 08:48:03.085075 14616 raft_consensus.cc:2932] T 00000000000000000000000000000000 P aae8b08fd1ac4d2699d15a396e7ac8f2 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 383bca56225440c4966e85359729cd9a (127.4.159.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "82a928c4f0234124b0567479c875bb25" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 37483 } } peers { permanent_uuid: "aae8b08fd1ac4d2699d15a396e7ac8f2" member_type: VOTER last_known_addr { host: "127.4.159.189" port: 37399 } attrs { promote: false } } peers { permanent_uuid: "383bca56225440c4966e85359729cd9a" member_type: VOTER last_known_addr { host: "127.4.159.188" port: 35089 } attrs { promote: false } } }
I20260807 08:48:03.085110 14900 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 383bca56225440c4966e85359729cd9a [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 383bca56225440c4966e85359729cd9a (127.4.159.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "82a928c4f0234124b0567479c875bb25" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 37483 } } peers { permanent_uuid: "aae8b08fd1ac4d2699d15a396e7ac8f2" member_type: VOTER last_known_addr { host: "127.4.159.189" port: 37399 } attrs { promote: false } } peers { permanent_uuid: "383bca56225440c4966e85359729cd9a" member_type: VOTER last_known_addr { host: "127.4.159.188" port: 35089 } attrs { promote: false } } }
I20260807 08:48:03.085695 14837 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82a928c4f0234124b0567479c875bb25 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "82a928c4f0234124b0567479c875bb25" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "82a928c4f0234124b0567479c875bb25" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 37483 } } peers { permanent_uuid: "aae8b08fd1ac4d2699d15a396e7ac8f2" member_type: VOTER last_known_addr { host: "127.4.159.189" port: 37399 } attrs { promote: false } } peers { permanent_uuid: "383bca56225440c4966e85359729cd9a" member_type: VOTER last_known_addr { host: "127.4.159.188" port: 35089 } attrs { promote: false } } }
I20260807 08:48:03.085772 14837 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82a928c4f0234124b0567479c875bb25 [sys.catalog]: This master's current role is: LEADER
I20260807 08:48:03.085820 14837 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82a928c4f0234124b0567479c875bb25 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "82a928c4f0234124b0567479c875bb25" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "82a928c4f0234124b0567479c875bb25" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 37483 } } peers { permanent_uuid: "aae8b08fd1ac4d2699d15a396e7ac8f2" member_type: VOTER last_known_addr { host: "127.4.159.189" port: 37399 } attrs { promote: false } } peers { permanent_uuid: "383bca56225440c4966e85359729cd9a" member_type: VOTER last_known_addr { host: "127.4.159.188" port: 35089 } attrs { promote: false } } }
I20260807 08:48:03.085866 14837 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82a928c4f0234124b0567479c875bb25 [sys.catalog]: This master's current role is: LEADER
I20260807 08:48:03.085897 14837 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82a928c4f0234124b0567479c875bb25 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "82a928c4f0234124b0567479c875bb25" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "82a928c4f0234124b0567479c875bb25" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 37483 } } peers { permanent_uuid: "aae8b08fd1ac4d2699d15a396e7ac8f2" member_type: VOTER last_known_addr { host: "127.4.159.189" port: 37399 } attrs { promote: false } } peers { permanent_uuid: "383bca56225440c4966e85359729cd9a" member_type: VOTER last_known_addr { host: "127.4.159.188" port: 35089 } attrs { promote: false } } }
I20260807 08:48:03.085939 14837 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82a928c4f0234124b0567479c875bb25 [sys.catalog]: This master's current role is: LEADER
I20260807 08:48:03.086014 14951 sys_catalog.cc:455] T 00000000000000000000000000000000 P 383bca56225440c4966e85359729cd9a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "82a928c4f0234124b0567479c875bb25" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "82a928c4f0234124b0567479c875bb25" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 37483 } } peers { permanent_uuid: "aae8b08fd1ac4d2699d15a396e7ac8f2" member_type: VOTER last_known_addr { host: "127.4.159.189" port: 37399 } attrs { promote: false } } peers { permanent_uuid: "383bca56225440c4966e85359729cd9a" member_type: VOTER last_known_addr { host: "127.4.159.188" port: 35089 } attrs { promote: false } } }
I20260807 08:48:03.086110 14951 sys_catalog.cc:458] T 00000000000000000000000000000000 P 383bca56225440c4966e85359729cd9a [sys.catalog]: This master's current role is: FOLLOWER
I20260807 08:48:03.086812 15631 sys_catalog.cc:455] T 00000000000000000000000000000000 P aae8b08fd1ac4d2699d15a396e7ac8f2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "82a928c4f0234124b0567479c875bb25" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "82a928c4f0234124b0567479c875bb25" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 37483 } } peers { permanent_uuid: "aae8b08fd1ac4d2699d15a396e7ac8f2" member_type: VOTER last_known_addr { host: "127.4.159.189" port: 37399 } attrs { promote: false } } peers { permanent_uuid: "383bca56225440c4966e85359729cd9a" member_type: VOTER last_known_addr { host: "127.4.159.188" port: 35089 } attrs { promote: false } } }
I20260807 08:48:03.086894 15631 sys_catalog.cc:458] T 00000000000000000000000000000000 P aae8b08fd1ac4d2699d15a396e7ac8f2 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 08:48:03.090303 14527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:03.086939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.130:39079 (local address 127.0.0.1:44222)
0807 08:48:03.086978 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:03.086989 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:03.087031 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:03.087535 (+ 504us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:03.087536 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:03.087538 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:48:03.087571 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:03.087572 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:03.088876 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:03.088877 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:03.089553 (+ 676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:03.089555 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:03.090170 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:03.090171 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:03.090211 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:03.090213 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 08:48:03.090237 (+ 24us) client_negotiation.cc:240] Negotiation successful
0807 08:48:03.090262 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21}
I20260807 08:48:03.090427 15633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:03.087041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44222 (local address 127.4.159.130:39079)
0807 08:48:03.087345 (+ 304us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:03.087350 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:03.087359 (+ 9us) server_negotiation.cc:407] Connection header received
0807 08:48:03.087399 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:03.087400 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:03.087416 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:03.087641 (+ 225us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:48:03.087702 (+ 61us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:03.088783 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:03.089629 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:03.090103 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:03.090165 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:03.090244 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 08:48:03.090270 (+ 26us) server_negotiation.cc:299] Negotiation successful
0807 08:48:03.090306 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":268,"thread_start_us":79,"threads_started":1}
I20260807 08:48:03.091054 15279 tablet_service.cc:1467] Processing CreateTablet for tablet c079ec1f40ae427387abd8045bb2f4ef (DEFAULT_TABLE table=test-table [id=1af54328465f40159b1877eb1f830da5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:48:03.091166 15632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:03.087031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.129:35375 (local address 127.0.0.1:60114)
0807 08:48:03.087165 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:03.087174 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:03.087229 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:03.087548 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:03.087549 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:03.087550 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:48:03.087587 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:03.087588 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:03.088825 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:03.088826 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:03.090004 (+ 1178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:03.090006 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:03.090937 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:03.090938 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:03.091004 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:03.091007 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 08:48:03.091049 (+ 42us) client_negotiation.cc:240] Negotiation successful
0807 08:48:03.091089 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"mutex_wait_us":55,"thread_start_us":90,"threads_started":1}
I20260807 08:48:03.091328 15634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:03.087180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60114 (local address 127.4.159.129:35375)
0807 08:48:03.087359 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:03.087362 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:03.087371 (+ 9us) server_negotiation.cc:407] Connection header received
0807 08:48:03.087405 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:03.087406 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:03.087420 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:03.087489 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:48:03.087657 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:03.088745 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:03.090086 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:03.090797 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:03.090857 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:03.091185 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0807 08:48:03.091216 (+ 31us) server_negotiation.cc:299] Negotiation successful
0807 08:48:03.091255 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":78,"threads_started":1}
I20260807 08:48:03.091399 15279 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c079ec1f40ae427387abd8045bb2f4ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:48:03.091976 15067 tablet_service.cc:1467] Processing CreateTablet for tablet c079ec1f40ae427387abd8045bb2f4ef (DEFAULT_TABLE table=test-table [id=1af54328465f40159b1877eb1f830da5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:48:03.092252 15635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:03.087285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.131:43209 (local address 127.0.0.1:53508)
0807 08:48:03.087578 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:03.087587 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:03.087636 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:03.087995 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:03.087996 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:03.087998 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:48:03.088033 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:03.088034 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:03.088923 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:03.088924 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:03.090001 (+ 1077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:03.090004 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:03.092016 (+ 2012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:03.092017 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:03.092075 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:03.092079 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 08:48:03.092137 (+ 58us) client_negotiation.cc:240] Negotiation successful
0807 08:48:03.092177 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"mutex_wait_us":1,"thread_start_us":67,"threads_started":1}
I20260807 08:48:03.092315 15067 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c079ec1f40ae427387abd8045bb2f4ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:48:03.092422 15636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:03.087344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53508 (local address 127.4.159.131:43209)
0807 08:48:03.087532 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:03.087536 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:03.087807 (+ 271us) server_negotiation.cc:407] Connection header received
0807 08:48:03.087909 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:03.087911 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:03.087926 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:03.087991 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:48:03.088080 (+ 89us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:03.088839 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:03.091422 (+ 2583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:03.091938 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:03.092002 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:03.092069 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 08:48:03.092154 (+ 85us) server_negotiation.cc:299] Negotiation successful
0807 08:48:03.092376 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":60,"threads_started":1}
I20260807 08:48:03.094846 15638 tablet_bootstrap.cc:492] T c079ec1f40ae427387abd8045bb2f4ef P bed5f8e9f0ba46ea97101418a4f0affe: Bootstrap starting.
I20260807 08:48:03.094887 15488 tablet_service.cc:1467] Processing CreateTablet for tablet c079ec1f40ae427387abd8045bb2f4ef (DEFAULT_TABLE table=test-table [id=1af54328465f40159b1877eb1f830da5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:48:03.095181 15488 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c079ec1f40ae427387abd8045bb2f4ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:48:03.095446 15638 tablet_bootstrap.cc:654] T c079ec1f40ae427387abd8045bb2f4ef P bed5f8e9f0ba46ea97101418a4f0affe: Neither blocks nor log segments found. Creating new log.
I20260807 08:48:03.095597 15641 tablet_bootstrap.cc:492] T c079ec1f40ae427387abd8045bb2f4ef P 67262c924f4e4493bd8dcab7c92e6aaa: Bootstrap starting.
I20260807 08:48:03.095763 15638 log.cc:824] T c079ec1f40ae427387abd8045bb2f4ef P bed5f8e9f0ba46ea97101418a4f0affe: Log is configured to *not* fsync() on all Append() calls
I20260807 08:48:03.096196 15641 tablet_bootstrap.cc:654] T c079ec1f40ae427387abd8045bb2f4ef P 67262c924f4e4493bd8dcab7c92e6aaa: Neither blocks nor log segments found. Creating new log.
I20260807 08:48:03.096468 15641 log.cc:824] T c079ec1f40ae427387abd8045bb2f4ef P 67262c924f4e4493bd8dcab7c92e6aaa: Log is configured to *not* fsync() on all Append() calls
I20260807 08:48:03.096570 15638 tablet_bootstrap.cc:492] T c079ec1f40ae427387abd8045bb2f4ef P bed5f8e9f0ba46ea97101418a4f0affe: No bootstrap required, opened a new log
I20260807 08:48:03.096642 15638 ts_tablet_manager.cc:1397] T c079ec1f40ae427387abd8045bb2f4ef P bed5f8e9f0ba46ea97101418a4f0affe: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 08:48:03.097209 15641 tablet_bootstrap.cc:492] T c079ec1f40ae427387abd8045bb2f4ef P 67262c924f4e4493bd8dcab7c92e6aaa: No bootstrap required, opened a new log
I20260807 08:48:03.097270 15641 ts_tablet_manager.cc:1397] T c079ec1f40ae427387abd8045bb2f4ef P 67262c924f4e4493bd8dcab7c92e6aaa: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 08:48:03.098373 15645 tablet_bootstrap.cc:492] T c079ec1f40ae427387abd8045bb2f4ef P 48c69edfbb324b37ace3f551898f2220: Bootstrap starting.
I20260807 08:48:03.098743 15641 raft_consensus.cc:348] T c079ec1f40ae427387abd8045bb2f4ef P 67262c924f4e4493bd8dcab7c92e6aaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bed5f8e9f0ba46ea97101418a4f0affe" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 39079 } } peers { permanent_uuid: "67262c924f4e4493bd8dcab7c92e6aaa" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 35375 } } peers { permanent_uuid: "48c69edfbb324b37ace3f551898f2220" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 43209 } }
I20260807 08:48:03.098906 15641 raft_consensus.cc:374] T c079ec1f40ae427387abd8045bb2f4ef P 67262c924f4e4493bd8dcab7c92e6aaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:48:03.098943 15641 raft_consensus.cc:729] T c079ec1f40ae427387abd8045bb2f4ef P 67262c924f4e4493bd8dcab7c92e6aaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67262c924f4e4493bd8dcab7c92e6aaa, State: Initialized, Role: FOLLOWER
I20260807 08:48:03.099042 15645 tablet_bootstrap.cc:654] T c079ec1f40ae427387abd8045bb2f4ef P 48c69edfbb324b37ace3f551898f2220: Neither blocks nor log segments found. Creating new log.
I20260807 08:48:03.099062 15641 consensus_queue.cc:260] T c079ec1f40ae427387abd8045bb2f4ef P 67262c924f4e4493bd8dcab7c92e6aaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bed5f8e9f0ba46ea97101418a4f0affe" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 39079 } } peers { permanent_uuid: "67262c924f4e4493bd8dcab7c92e6aaa" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 35375 } } peers { permanent_uuid: "48c69edfbb324b37ace3f551898f2220" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 43209 } }
I20260807 08:48:03.099099 15638 raft_consensus.cc:348] T c079ec1f40ae427387abd8045bb2f4ef P bed5f8e9f0ba46ea97101418a4f0affe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bed5f8e9f0ba46ea97101418a4f0affe" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 39079 } } peers { permanent_uuid: "67262c924f4e4493bd8dcab7c92e6aaa" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 35375 } } peers { permanent_uuid: "48c69edfbb324b37ace3f551898f2220" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 43209 } }
I20260807 08:48:03.099254 15638 raft_consensus.cc:374] T c079ec1f40ae427387abd8045bb2f4ef P bed5f8e9f0ba46ea97101418a4f0affe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:48:03.099301 15638 raft_consensus.cc:729] T c079ec1f40ae427387abd8045bb2f4ef P bed5f8e9f0ba46ea97101418a4f0affe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bed5f8e9f0ba46ea97101418a4f0affe, State: Initialized, Role: FOLLOWER
I20260807 08:48:03.099334 15641 ts_tablet_manager.cc:1428] T c079ec1f40ae427387abd8045bb2f4ef P 67262c924f4e4493bd8dcab7c92e6aaa: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 08:48:03.099349 15645 log.cc:824] T c079ec1f40ae427387abd8045bb2f4ef P 48c69edfbb324b37ace3f551898f2220: Log is configured to *not* fsync() on all Append() calls
I20260807 08:48:03.099444 15167 heartbeater.cc:503] Master 127.4.159.190:37483 was elected leader, sending a full tablet report...
I20260807 08:48:03.099429 15638 consensus_queue.cc:260] T c079ec1f40ae427387abd8045bb2f4ef P bed5f8e9f0ba46ea97101418a4f0affe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bed5f8e9f0ba46ea97101418a4f0affe" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 39079 } } peers { permanent_uuid: "67262c924f4e4493bd8dcab7c92e6aaa" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 35375 } } peers { permanent_uuid: "48c69edfbb324b37ace3f551898f2220" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 43209 } }
I20260807 08:48:03.100096 15645 tablet_bootstrap.cc:492] T c079ec1f40ae427387abd8045bb2f4ef P 48c69edfbb324b37ace3f551898f2220: No bootstrap required, opened a new log
I20260807 08:48:03.100160 15645 ts_tablet_manager.cc:1397] T c079ec1f40ae427387abd8045bb2f4ef P 48c69edfbb324b37ace3f551898f2220: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 08:48:03.100477 15638 ts_tablet_manager.cc:1428] T c079ec1f40ae427387abd8045bb2f4ef P bed5f8e9f0ba46ea97101418a4f0affe: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260807 08:48:03.100587 15389 heartbeater.cc:503] Master 127.4.159.190:37483 was elected leader, sending a full tablet report...
I20260807 08:48:03.102313 15645 raft_consensus.cc:348] T c079ec1f40ae427387abd8045bb2f4ef P 48c69edfbb324b37ace3f551898f2220 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bed5f8e9f0ba46ea97101418a4f0affe" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 39079 } } peers { permanent_uuid: "67262c924f4e4493bd8dcab7c92e6aaa" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 35375 } } peers { permanent_uuid: "48c69edfbb324b37ace3f551898f2220" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 43209 } }
I20260807 08:48:03.102475 15645 raft_consensus.cc:374] T c079ec1f40ae427387abd8045bb2f4ef P 48c69edfbb324b37ace3f551898f2220 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:48:03.102516 15645 raft_consensus.cc:729] T c079ec1f40ae427387abd8045bb2f4ef P 48c69edfbb324b37ace3f551898f2220 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48c69edfbb324b37ace3f551898f2220, State: Initialized, Role: FOLLOWER
I20260807 08:48:03.102618 15645 consensus_queue.cc:260] T c079ec1f40ae427387abd8045bb2f4ef P 48c69edfbb324b37ace3f551898f2220 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bed5f8e9f0ba46ea97101418a4f0affe" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 39079 } } peers { permanent_uuid: "67262c924f4e4493bd8dcab7c92e6aaa" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 35375 } } peers { permanent_uuid: "48c69edfbb324b37ace3f551898f2220" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 43209 } }
I20260807 08:48:03.102833 15645 ts_tablet_manager.cc:1428] T c079ec1f40ae427387abd8045bb2f4ef P 48c69edfbb324b37ace3f551898f2220: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260807 08:48:03.102869 15583 heartbeater.cc:503] Master 127.4.159.190:37483 was elected leader, sending a full tablet report...
W20260807 08:48:03.183164 15393 tablet.cc:2369] T c079ec1f40ae427387abd8045bb2f4ef P bed5f8e9f0ba46ea97101418a4f0affe: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 08:48:03.263609 15648 raft_consensus.cc:482] T c079ec1f40ae427387abd8045bb2f4ef P 67262c924f4e4493bd8dcab7c92e6aaa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:48:03.263724 15648 raft_consensus.cc:504] T c079ec1f40ae427387abd8045bb2f4ef P 67262c924f4e4493bd8dcab7c92e6aaa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bed5f8e9f0ba46ea97101418a4f0affe" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 39079 } } peers { permanent_uuid: "67262c924f4e4493bd8dcab7c92e6aaa" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 35375 } } peers { permanent_uuid: "48c69edfbb324b37ace3f551898f2220" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 43209 } }
I20260807 08:48:03.264073 15648 leader_election.cc:288] T c079ec1f40ae427387abd8045bb2f4ef P 67262c924f4e4493bd8dcab7c92e6aaa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bed5f8e9f0ba46ea97101418a4f0affe (127.4.159.130:39079), 48c69edfbb324b37ace3f551898f2220 (127.4.159.131:43209)
I20260807 08:48:03.267056 15172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:03.264274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.131:43209 (local address 127.4.159.129:52995)
0807 08:48:03.264327 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:03.264339 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:03.264393 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:03.264558 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:03.264558 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:03.264560 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:48:03.264632 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:03.264634 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:03.265408 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:03.265409 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:03.266259 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:03.266262 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:03.266835 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:03.266836 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:03.266909 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:03.266914 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 08:48:03.266966 (+ 52us) client_negotiation.cc:240] Negotiation successful
0807 08:48:03.267002 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260807 08:48:03.267083 15636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:03.264404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.129:52995 (local address 127.4.159.131:43209)
0807 08:48:03.264458 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:03.264460 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:03.264468 (+ 8us) server_negotiation.cc:407] Connection header received
0807 08:48:03.264492 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:03.264493 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:03.264501 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:03.264541 (+ 40us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:48:03.264693 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:03.265337 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:03.266361 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:03.266768 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:03.266820 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:03.266828 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0807 08:48:03.266976 (+ 148us) server_negotiation.cc:299] Negotiation successful
0807 08:48:03.267013 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260807 08:48:03.267242 15173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:03.264274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.130:39079 (local address 127.4.159.129:45323)
0807 08:48:03.264363 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:03.264373 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:03.264412 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:03.264588 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:03.264589 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:03.264591 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:48:03.264972 (+ 381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:03.264975 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:03.265641 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:03.265642 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:03.266440 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:03.266443 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:03.266979 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:03.266980 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:03.267070 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:03.267074 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 08:48:03.267126 (+ 52us) client_negotiation.cc:240] Negotiation successful
0807 08:48:03.267163 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":71,"mutex_wait_us":51}
I20260807 08:48:03.267405 15518 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c079ec1f40ae427387abd8045bb2f4ef" candidate_uuid: "67262c924f4e4493bd8dcab7c92e6aaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48c69edfbb324b37ace3f551898f2220" is_pre_election: true
I20260807 08:48:03.267489 15633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:03.264432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.129:45323 (local address 127.4.159.130:39079)
0807 08:48:03.264475 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:03.264476 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:03.264484 (+ 8us) server_negotiation.cc:407] Connection header received
0807 08:48:03.264505 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:03.264506 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:03.264513 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:03.264563 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:48:03.265048 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:03.265556 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:03.266520 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:03.266898 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:03.266969 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:03.266977 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0807 08:48:03.267396 (+ 419us) server_negotiation.cc:299] Negotiation successful
0807 08:48:03.267442 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 08:48:03.267529 15518 raft_consensus.cc:2454] T c079ec1f40ae427387abd8045bb2f4ef P 48c69edfbb324b37ace3f551898f2220 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 67262c924f4e4493bd8dcab7c92e6aaa in term 0.
I20260807 08:48:03.267750 14994 leader_election.cc:302] T c079ec1f40ae427387abd8045bb2f4ef P 67262c924f4e4493bd8dcab7c92e6aaa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48c69edfbb324b37ace3f551898f2220, 67262c924f4e4493bd8dcab7c92e6aaa; no voters:
I20260807 08:48:03.267778 15312 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c079ec1f40ae427387abd8045bb2f4ef" candidate_uuid: "67262c924f4e4493bd8dcab7c92e6aaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bed5f8e9f0ba46ea97101418a4f0affe" is_pre_election: true
I20260807 08:48:03.267869 15648 raft_consensus.cc:2781] T c079ec1f40ae427387abd8045bb2f4ef P 67262c924f4e4493bd8dcab7c92e6aaa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:48:03.267905 15312 raft_consensus.cc:2454] T c079ec1f40ae427387abd8045bb2f4ef P bed5f8e9f0ba46ea97101418a4f0affe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 67262c924f4e4493bd8dcab7c92e6aaa in term 0.
I20260807 08:48:03.267925 15648 raft_consensus.cc:482] T c079ec1f40ae427387abd8045bb2f4ef P 67262c924f4e4493bd8dcab7c92e6aaa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:48:03.267947 15648 raft_consensus.cc:3037] T c079ec1f40ae427387abd8045bb2f4ef P 67262c924f4e4493bd8dcab7c92e6aaa [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:48:03.268676 15648 raft_consensus.cc:504] T c079ec1f40ae427387abd8045bb2f4ef P 67262c924f4e4493bd8dcab7c92e6aaa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bed5f8e9f0ba46ea97101418a4f0affe" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 39079 } } peers { permanent_uuid: "67262c924f4e4493bd8dcab7c92e6aaa" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 35375 } } peers { permanent_uuid: "48c69edfbb324b37ace3f551898f2220" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 43209 } }
I20260807 08:48:03.268800 15648 leader_election.cc:288] T c079ec1f40ae427387abd8045bb2f4ef P 67262c924f4e4493bd8dcab7c92e6aaa [CANDIDATE]: Term 1 election: Requested vote from peers bed5f8e9f0ba46ea97101418a4f0affe (127.4.159.130:39079), 48c69edfbb324b37ace3f551898f2220 (127.4.159.131:43209)
I20260807 08:48:03.268940 15312 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c079ec1f40ae427387abd8045bb2f4ef" candidate_uuid: "67262c924f4e4493bd8dcab7c92e6aaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bed5f8e9f0ba46ea97101418a4f0affe"
I20260807 08:48:03.268940 15518 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c079ec1f40ae427387abd8045bb2f4ef" candidate_uuid: "67262c924f4e4493bd8dcab7c92e6aaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48c69edfbb324b37ace3f551898f2220"
I20260807 08:48:03.269006 15312 raft_consensus.cc:3037] T c079ec1f40ae427387abd8045bb2f4ef P bed5f8e9f0ba46ea97101418a4f0affe [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:48:03.269006 15518 raft_consensus.cc:3037] T c079ec1f40ae427387abd8045bb2f4ef P 48c69edfbb324b37ace3f551898f2220 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:48:03.269833 15518 raft_consensus.cc:2454] T c079ec1f40ae427387abd8045bb2f4ef P 48c69edfbb324b37ace3f551898f2220 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67262c924f4e4493bd8dcab7c92e6aaa in term 1.
I20260807 08:48:03.269833 15312 raft_consensus.cc:2454] T c079ec1f40ae427387abd8045bb2f4ef P bed5f8e9f0ba46ea97101418a4f0affe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67262c924f4e4493bd8dcab7c92e6aaa in term 1.
I20260807 08:48:03.269987 14997 leader_election.cc:302] T c079ec1f40ae427387abd8045bb2f4ef P 67262c924f4e4493bd8dcab7c92e6aaa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 67262c924f4e4493bd8dcab7c92e6aaa, bed5f8e9f0ba46ea97101418a4f0affe; no voters:
I20260807 08:48:03.270077 15648 raft_consensus.cc:2781] T c079ec1f40ae427387abd8045bb2f4ef P 67262c924f4e4493bd8dcab7c92e6aaa [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:48:03.270201 15648 raft_consensus.cc:686] T c079ec1f40ae427387abd8045bb2f4ef P 67262c924f4e4493bd8dcab7c92e6aaa [term 1 LEADER]: Becoming Leader. State: Replica: 67262c924f4e4493bd8dcab7c92e6aaa, State: Running, Role: LEADER
I20260807 08:48:03.270386 15648 consensus_queue.cc:237] T c079ec1f40ae427387abd8045bb2f4ef P 67262c924f4e4493bd8dcab7c92e6aaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bed5f8e9f0ba46ea97101418a4f0affe" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 39079 } } peers { permanent_uuid: "67262c924f4e4493bd8dcab7c92e6aaa" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 35375 } } peers { permanent_uuid: "48c69edfbb324b37ace3f551898f2220" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 43209 } }
I20260807 08:48:03.271394 13653 catalog_manager.cc:5495] T c079ec1f40ae427387abd8045bb2f4ef P 67262c924f4e4493bd8dcab7c92e6aaa reported cstate change: term changed from 0 to 1, leader changed from <none> to 67262c924f4e4493bd8dcab7c92e6aaa (127.4.159.129). New cstate: current_term: 1 leader_uuid: "67262c924f4e4493bd8dcab7c92e6aaa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bed5f8e9f0ba46ea97101418a4f0affe" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 39079 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "67262c924f4e4493bd8dcab7c92e6aaa" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 35375 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "48c69edfbb324b37ace3f551898f2220" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 43209 } health_report { overall_health: UNKNOWN } } }
W20260807 08:48:03.278440 15171 tablet.cc:2369] T c079ec1f40ae427387abd8045bb2f4ef P 67262c924f4e4493bd8dcab7c92e6aaa: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 08:48:03.312462 15634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:03.310435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60116 (local address 127.4.159.129:35375)
0807 08:48:03.310497 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:03.310499 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:03.310530 (+ 31us) server_negotiation.cc:407] Connection header received
0807 08:48:03.310566 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:03.310566 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:03.310584 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:03.310634 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:48:03.310738 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:03.311181 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:03.311874 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:03.312063 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:03.312109 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:03.312224 (+ 115us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:48:03.312319 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:48:03.312371 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0807 08:48:03.312387 (+ 16us) server_negotiation.cc:299] Negotiation successful
0807 08:48:03.312403 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
W20260807 08:48:03.312537 15587 tablet.cc:2369] T c079ec1f40ae427387abd8045bb2f4ef P 48c69edfbb324b37ace3f551898f2220: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 08:48:03.314116 15518 raft_consensus.cc:1261] T c079ec1f40ae427387abd8045bb2f4ef P 48c69edfbb324b37ace3f551898f2220 [term 1 FOLLOWER]: Refusing update from remote peer 67262c924f4e4493bd8dcab7c92e6aaa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:48:03.314448 15312 raft_consensus.cc:1261] T c079ec1f40ae427387abd8045bb2f4ef P bed5f8e9f0ba46ea97101418a4f0affe [term 1 FOLLOWER]: Refusing update from remote peer 67262c924f4e4493bd8dcab7c92e6aaa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:48:03.314476 15653 consensus_queue.cc:1035] T c079ec1f40ae427387abd8045bb2f4ef P 67262c924f4e4493bd8dcab7c92e6aaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "48c69edfbb324b37ace3f551898f2220" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 43209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:48:03.314714 15653 consensus_queue.cc:1035] T c079ec1f40ae427387abd8045bb2f4ef P 67262c924f4e4493bd8dcab7c92e6aaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "bed5f8e9f0ba46ea97101418a4f0affe" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 39079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:48:03.318384 15657 mvcc.cc:205] Tried to move back new op lower bound from 7315834811652067328 to 7315834811476205568. Current Snapshot: MvccSnapshot[applied={T|T < 7315834811652067328}]
I20260807 08:48:03.318702 15659 mvcc.cc:205] Tried to move back new op lower bound from 7315834811652067328 to 7315834811476205568. Current Snapshot: MvccSnapshot[applied={T|T < 7315834811652067328}]
I20260807 08:48:03.319912 13653 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44882:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:48:03.319996 13653 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44882:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:48:03.321614 13653 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 82a928c4f0234124b0567479c875bb25: Sending DeleteTablet for 3 replicas of tablet c079ec1f40ae427387abd8045bb2f4ef
I20260807 08:48:03.321991 15488 tablet_service.cc:1514] Processing DeleteTablet for tablet c079ec1f40ae427387abd8045bb2f4ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:48:03 UTC) from {username='oryx', principal='oryx/127.4.159.190@KRBTEST.COM'} at 127.0.0.1:53508
I20260807 08:48:03.321997 15067 tablet_service.cc:1514] Processing DeleteTablet for tablet c079ec1f40ae427387abd8045bb2f4ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:48:03 UTC) from {username='oryx', principal='oryx/127.4.159.190@KRBTEST.COM'} at 127.0.0.1:60114
I20260807 08:48:03.322288 15279 tablet_service.cc:1514] Processing DeleteTablet for tablet c079ec1f40ae427387abd8045bb2f4ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:48:03 UTC) from {username='oryx', principal='oryx/127.4.159.190@KRBTEST.COM'} at 127.0.0.1:44222
I20260807 08:48:03.322454 15669 tablet_replica.cc:330] T c079ec1f40ae427387abd8045bb2f4ef P 48c69edfbb324b37ace3f551898f2220: stopping tablet replica
I20260807 08:48:03.322546 15669 raft_consensus.cc:2229] T c079ec1f40ae427387abd8045bb2f4ef P 48c69edfbb324b37ace3f551898f2220 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:48:03.322629 15670 tablet_replica.cc:330] T c079ec1f40ae427387abd8045bb2f4ef P bed5f8e9f0ba46ea97101418a4f0affe: stopping tablet replica
I20260807 08:48:03.322650 15669 raft_consensus.cc:2258] T c079ec1f40ae427387abd8045bb2f4ef P 48c69edfbb324b37ace3f551898f2220 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:48:03.322777 15670 raft_consensus.cc:2229] T c079ec1f40ae427387abd8045bb2f4ef P bed5f8e9f0ba46ea97101418a4f0affe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:48:03.322923 15670 raft_consensus.cc:2258] T c079ec1f40ae427387abd8045bb2f4ef P bed5f8e9f0ba46ea97101418a4f0affe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:48:03.323093 15669 ts_tablet_manager.cc:1905] T c079ec1f40ae427387abd8045bb2f4ef P 48c69edfbb324b37ace3f551898f2220: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:48:03.323340 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 13603
I20260807 08:48:03.323474 15670 ts_tablet_manager.cc:1905] T c079ec1f40ae427387abd8045bb2f4ef P bed5f8e9f0ba46ea97101418a4f0affe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:48:03.325212 15670 ts_tablet_manager.cc:1918] T c079ec1f40ae427387abd8045bb2f4ef P bed5f8e9f0ba46ea97101418a4f0affe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:48:03.325284 15670 log.cc:1196] T c079ec1f40ae427387abd8045bb2f4ef P bed5f8e9f0ba46ea97101418a4f0affe: Deleting WAL directory at /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/ts-1/wal/wals/c079ec1f40ae427387abd8045bb2f4ef
I20260807 08:48:03.325589 15670 ts_tablet_manager.cc:1939] T c079ec1f40ae427387abd8045bb2f4ef P bed5f8e9f0ba46ea97101418a4f0affe: Deleting consensus metadata
I20260807 08:48:03.326025 15668 tablet_replica.cc:330] T c079ec1f40ae427387abd8045bb2f4ef P 67262c924f4e4493bd8dcab7c92e6aaa: stopping tablet replica
I20260807 08:48:03.326122 15668 raft_consensus.cc:2229] T c079ec1f40ae427387abd8045bb2f4ef P 67262c924f4e4493bd8dcab7c92e6aaa [term 1 LEADER]: Raft consensus shutting down.
I20260807 08:48:03.326220 15668 raft_consensus.cc:2258] T c079ec1f40ae427387abd8045bb2f4ef P 67262c924f4e4493bd8dcab7c92e6aaa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:48:03.328850 15668 ts_tablet_manager.cc:1905] T c079ec1f40ae427387abd8045bb2f4ef P 67262c924f4e4493bd8dcab7c92e6aaa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:48:03.331061 15668 ts_tablet_manager.cc:1918] T c079ec1f40ae427387abd8045bb2f4ef P 67262c924f4e4493bd8dcab7c92e6aaa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:48:03.331135 15668 log.cc:1196] T c079ec1f40ae427387abd8045bb2f4ef P 67262c924f4e4493bd8dcab7c92e6aaa: Deleting WAL directory at /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/ts-0/wal/wals/c079ec1f40ae427387abd8045bb2f4ef
I20260807 08:48:03.331519 15668 ts_tablet_manager.cc:1939] T c079ec1f40ae427387abd8045bb2f4ef P 67262c924f4e4493bd8dcab7c92e6aaa: Deleting consensus metadata
I20260807 08:48:03.333843 15669 ts_tablet_manager.cc:1918] T c079ec1f40ae427387abd8045bb2f4ef P 48c69edfbb324b37ace3f551898f2220: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:48:03.333918 15669 log.cc:1196] T c079ec1f40ae427387abd8045bb2f4ef P 48c69edfbb324b37ace3f551898f2220: Deleting WAL directory at /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/ts-2/wal/wals/c079ec1f40ae427387abd8045bb2f4ef
I20260807 08:48:03.334152 15669 ts_tablet_manager.cc:1939] T c079ec1f40ae427387abd8045bb2f4ef P 48c69edfbb324b37ace3f551898f2220: Deleting consensus metadata
W20260807 08:48:03.339255 15423 connection.cc:535] server connection from 127.0.0.1:53508 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260807 08:48:03.339279 14996 connection.cc:535] server connection from 127.0.0.1:60114 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260807 08:48:03.339705 15208 connection.cc:535] server connection from 127.0.0.1:44222 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 08:48:03.347934 15608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:03.340920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57408 (local address 127.4.159.188:35089)
0807 08:48:03.340959 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:03.340961 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:03.341002 (+ 41us) server_negotiation.cc:407] Connection header received
0807 08:48:03.341055 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:03.341056 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:03.341067 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:03.341123 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:03.341659 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:03.342422 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:03.343489 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:03.343733 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:03.343782 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:03.345280 (+ 1498us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:48:03.345281 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:48:03.345281 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:48:03.345288 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:48:03.346766 (+ 1478us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:48:03.347153 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:48:03.347154 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:48:03.347154 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:48:03.347214 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:48:03.347397 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:48:03.347398 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:48:03.347398 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:48:03.347513 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:48:03.347593 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 08:48:03.347720 (+ 127us) server_negotiation.cc:299] Negotiation successful
0807 08:48:03.347842 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 08:48:03.348013 15602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:03.340949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49276 (local address 127.4.159.189:37399)
0807 08:48:03.340991 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:03.340992 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:03.341000 (+ 8us) server_negotiation.cc:407] Connection header received
0807 08:48:03.341024 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:03.341025 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:03.341036 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:03.341119 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:03.341818 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:03.342550 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:03.343218 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:03.343427 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:03.343480 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:03.345055 (+ 1575us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:48:03.345057 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:48:03.345057 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:48:03.345064 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:48:03.346625 (+ 1561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:48:03.347043 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:48:03.347045 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:48:03.347045 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:48:03.347102 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:48:03.347298 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:48:03.347299 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:48:03.347299 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:48:03.347417 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:48:03.347513 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0807 08:48:03.347832 (+ 319us) server_negotiation.cc:299] Negotiation successful
0807 08:48:03.347918 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 08:48:04.318984 15679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:04.318697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:37483 (local address 127.4.159.131:46723)
0807 08:48:04.318836 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:04.318929 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:37483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":70,"threads_started":1}
W20260807 08:48:04.319238 15583 heartbeater.cc:644] Failed to heartbeat to 127.4.159.190:37483 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.4.159.190:37483: connect: Connection refused (error 111)
I20260807 08:48:04.319522 15680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:04.319305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:37483 (local address 127.4.159.129:42393)
0807 08:48:04.319428 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:04.319488 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:37483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":55,"threads_started":1}
I20260807 08:48:04.319640 15682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:04.319415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:37483 (local address 127.4.159.130:51381)
0807 08:48:04.319529 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:04.319597 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:37483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":53,"threads_started":1}
W20260807 08:48:04.319734 15167 heartbeater.cc:644] Failed to heartbeat to 127.4.159.190:37483 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.4.159.190:37483: connect: Connection refused (error 111)
W20260807 08:48:04.319880 15389 heartbeater.cc:644] Failed to heartbeat to 127.4.159.190:37483 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.4.159.190:37483: connect: Connection refused (error 111)
I20260807 08:48:04.822149 15683 raft_consensus.cc:482] T 00000000000000000000000000000000 P aae8b08fd1ac4d2699d15a396e7ac8f2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 82a928c4f0234124b0567479c875bb25)
I20260807 08:48:04.822294 15683 raft_consensus.cc:504] T 00000000000000000000000000000000 P aae8b08fd1ac4d2699d15a396e7ac8f2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "82a928c4f0234124b0567479c875bb25" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 37483 } } peers { permanent_uuid: "aae8b08fd1ac4d2699d15a396e7ac8f2" member_type: VOTER last_known_addr { host: "127.4.159.189" port: 37399 } attrs { promote: false } } peers { permanent_uuid: "383bca56225440c4966e85359729cd9a" member_type: VOTER last_known_addr { host: "127.4.159.188" port: 35089 } attrs { promote: false } }
I20260807 08:48:04.822568 15686 raft_consensus.cc:482] T 00000000000000000000000000000000 P 383bca56225440c4966e85359729cd9a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 82a928c4f0234124b0567479c875bb25)
I20260807 08:48:04.822583 15683 leader_election.cc:288] T 00000000000000000000000000000000 P aae8b08fd1ac4d2699d15a396e7ac8f2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 82a928c4f0234124b0567479c875bb25 (127.4.159.190:37483), 383bca56225440c4966e85359729cd9a (127.4.159.188:35089)
I20260807 08:48:04.822685 15686 raft_consensus.cc:504] T 00000000000000000000000000000000 P 383bca56225440c4966e85359729cd9a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "82a928c4f0234124b0567479c875bb25" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 37483 } } peers { permanent_uuid: "aae8b08fd1ac4d2699d15a396e7ac8f2" member_type: VOTER last_known_addr { host: "127.4.159.189" port: 37399 } attrs { promote: false } } peers { permanent_uuid: "383bca56225440c4966e85359729cd9a" member_type: VOTER last_known_addr { host: "127.4.159.188" port: 35089 } attrs { promote: false } }
I20260807 08:48:04.822926 15686 leader_election.cc:288] T 00000000000000000000000000000000 P 383bca56225440c4966e85359729cd9a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 82a928c4f0234124b0567479c875bb25 (127.4.159.190:37483), aae8b08fd1ac4d2699d15a396e7ac8f2 (127.4.159.189:37399)
I20260807 08:48:04.823165 15687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:04.822686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:37483 (local address 127.0.0.1:45370)
0807 08:48:04.822839 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:04.822893 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:37483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":49,"threads_started":1}
I20260807 08:48:04.823199 15690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:04.822978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:37483 (local address 127.0.0.1:45378)
0807 08:48:04.823095 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:04.823149 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:37483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":53,"threads_started":1}
W20260807 08:48:04.823334 14865 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.159.190:37483: connect: Connection refused (error 111)
W20260807 08:48:04.823400 14569 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.159.190:37483: connect: Connection refused (error 111)
I20260807 08:48:04.823860 15691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:04.823768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:37483 (local address 127.0.0.1:45382)
0807 08:48:04.823801 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:04.823830 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:37483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260807 08:48:04.823940 14865 leader_election.cc:334] T 00000000000000000000000000000000 P 383bca56225440c4966e85359729cd9a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 82a928c4f0234124b0567479c875bb25 (127.4.159.190:37483): Network error: Client connection negotiation failed: client connection to 127.4.159.190:37483: connect: Connection refused (error 111)
I20260807 08:48:04.823890 15687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:04.823734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:37483 (local address 127.0.0.1:45380)
0807 08:48:04.823841 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:04.823864 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:37483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":68,"threads_started":1}
W20260807 08:48:04.824146 14569 leader_election.cc:334] T 00000000000000000000000000000000 P aae8b08fd1ac4d2699d15a396e7ac8f2 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 82a928c4f0234124b0567479c875bb25 (127.4.159.190:37483): Network error: Client connection negotiation failed: client connection to 127.4.159.190:37483: connect: Connection refused (error 111)
I20260807 08:48:04.825736 15688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:04.822832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.188:35089 (local address 127.0.0.1:57416)
0807 08:48:04.823074 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:04.823085 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:04.823147 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:04.823312 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:04.823313 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:04.823315 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:48:04.823379 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:04.823381 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:04.824264 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:04.824265 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:04.825006 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:04.825008 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:04.825522 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:04.825523 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:04.825579 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:04.825583 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 08:48:04.825626 (+ 43us) client_negotiation.cc:240] Negotiation successful
0807 08:48:04.825670 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":36,"threads_started":1}
I20260807 08:48:04.825733 15689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:04.822962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57416 (local address 127.4.159.188:35089)
0807 08:48:04.823050 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:04.823053 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:04.823189 (+ 136us) server_negotiation.cc:407] Connection header received
0807 08:48:04.823217 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:04.823218 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:04.823229 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:04.823440 (+ 211us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:48:04.823477 (+ 37us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:04.824191 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:04.825084 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:04.825454 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:04.825514 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:04.825524 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0807 08:48:04.825636 (+ 112us) server_negotiation.cc:299] Negotiation successful
0807 08:48:04.825674 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68,"thread_start_us":48,"threads_started":1}
I20260807 08:48:04.825883 15692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:04.823075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49280 (local address 127.4.159.189:37399)
0807 08:48:04.823222 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:04.823224 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:04.823371 (+ 147us) server_negotiation.cc:407] Connection header received
0807 08:48:04.823395 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:04.823396 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:04.823406 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:04.823466 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:48:04.823635 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:04.824330 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:04.825227 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:04.825590 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:04.825647 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:04.825657 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0807 08:48:04.825777 (+ 120us) server_negotiation.cc:299] Negotiation successful
0807 08:48:04.825817 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":35,"threads_started":1}
I20260807 08:48:04.825958 14900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "aae8b08fd1ac4d2699d15a396e7ac8f2" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "383bca56225440c4966e85359729cd9a" is_pre_election: true
I20260807 08:48:04.826040 14900 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 383bca56225440c4966e85359729cd9a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aae8b08fd1ac4d2699d15a396e7ac8f2 in term 1.
I20260807 08:48:04.826193 14569 leader_election.cc:302] T 00000000000000000000000000000000 P aae8b08fd1ac4d2699d15a396e7ac8f2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 383bca56225440c4966e85359729cd9a, aae8b08fd1ac4d2699d15a396e7ac8f2; no voters: 82a928c4f0234124b0567479c875bb25
I20260807 08:48:04.826239 15690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:04.823043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.189:37399 (local address 127.0.0.1:49280)
0807 08:48:04.823255 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:04.823264 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:04.823321 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:04.823512 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:04.823513 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:04.823515 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:48:04.823567 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:04.823568 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:04.824410 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:04.824411 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:04.825151 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:04.825153 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:04.825655 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:04.825656 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:04.825721 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:04.825725 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 08:48:04.825768 (+ 43us) client_negotiation.cc:240] Negotiation successful
0807 08:48:04.826180 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":41,"threads_started":1}
I20260807 08:48:04.826329 15683 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aae8b08fd1ac4d2699d15a396e7ac8f2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 08:48:04.826380 15683 raft_consensus.cc:482] T 00000000000000000000000000000000 P aae8b08fd1ac4d2699d15a396e7ac8f2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 82a928c4f0234124b0567479c875bb25)
I20260807 08:48:04.826399 15683 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aae8b08fd1ac4d2699d15a396e7ac8f2 [term 1 FOLLOWER]: Advancing to term 2
I20260807 08:48:04.826462 14616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "383bca56225440c4966e85359729cd9a" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "aae8b08fd1ac4d2699d15a396e7ac8f2" is_pre_election: true
I20260807 08:48:04.827124 15683 raft_consensus.cc:504] T 00000000000000000000000000000000 P aae8b08fd1ac4d2699d15a396e7ac8f2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "82a928c4f0234124b0567479c875bb25" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 37483 } } peers { permanent_uuid: "aae8b08fd1ac4d2699d15a396e7ac8f2" member_type: VOTER last_known_addr { host: "127.4.159.189" port: 37399 } attrs { promote: false } } peers { permanent_uuid: "383bca56225440c4966e85359729cd9a" member_type: VOTER last_known_addr { host: "127.4.159.188" port: 35089 } attrs { promote: false } }
I20260807 08:48:04.827267 15683 leader_election.cc:288] T 00000000000000000000000000000000 P aae8b08fd1ac4d2699d15a396e7ac8f2 [CANDIDATE]: Term 2 election: Requested vote from peers 82a928c4f0234124b0567479c875bb25 (127.4.159.190:37483), 383bca56225440c4966e85359729cd9a (127.4.159.188:35089)
I20260807 08:48:04.827275 14616 raft_consensus.cc:2379] T 00000000000000000000000000000000 P aae8b08fd1ac4d2699d15a396e7ac8f2 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 383bca56225440c4966e85359729cd9a in current term 2: Already voted for candidate aae8b08fd1ac4d2699d15a396e7ac8f2 in this term.
I20260807 08:48:04.827430 15688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:04.827343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:37483 (local address 127.0.0.1:45386)
0807 08:48:04.827378 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:04.827407 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:37483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260807 08:48:04.827458 14900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "aae8b08fd1ac4d2699d15a396e7ac8f2" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "383bca56225440c4966e85359729cd9a"
I20260807 08:48:04.827512 14867 leader_election.cc:302] T 00000000000000000000000000000000 P 383bca56225440c4966e85359729cd9a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 383bca56225440c4966e85359729cd9a; no voters: 82a928c4f0234124b0567479c875bb25, aae8b08fd1ac4d2699d15a396e7ac8f2
I20260807 08:48:04.827538 14900 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 383bca56225440c4966e85359729cd9a [term 1 FOLLOWER]: Advancing to term 2
I20260807 08:48:04.828167 14900 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 383bca56225440c4966e85359729cd9a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aae8b08fd1ac4d2699d15a396e7ac8f2 in term 2.
I20260807 08:48:04.828265 15686 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 383bca56225440c4966e85359729cd9a [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 08:48:04.828469 14569 leader_election.cc:302] T 00000000000000000000000000000000 P aae8b08fd1ac4d2699d15a396e7ac8f2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 383bca56225440c4966e85359729cd9a, aae8b08fd1ac4d2699d15a396e7ac8f2; no voters:
I20260807 08:48:04.828523 15688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:04.828435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:37483 (local address 127.0.0.1:45400)
0807 08:48:04.828477 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:04.828504 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:37483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260807 08:48:04.828554 15683 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aae8b08fd1ac4d2699d15a396e7ac8f2 [term 2 FOLLOWER]: Leader election won for term 2
W20260807 08:48:04.828644 14569 leader_election.cc:334] T 00000000000000000000000000000000 P aae8b08fd1ac4d2699d15a396e7ac8f2 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 82a928c4f0234124b0567479c875bb25 (127.4.159.190:37483): Network error: Client connection negotiation failed: client connection to 127.4.159.190:37483: connect: Connection refused (error 111)
I20260807 08:48:04.828683 15683 raft_consensus.cc:686] T 00000000000000000000000000000000 P aae8b08fd1ac4d2699d15a396e7ac8f2 [term 2 LEADER]: Becoming Leader. State: Replica: aae8b08fd1ac4d2699d15a396e7ac8f2, State: Running, Role: LEADER
I20260807 08:48:04.828753 15683 consensus_queue.cc:237] T 00000000000000000000000000000000 P aae8b08fd1ac4d2699d15a396e7ac8f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "82a928c4f0234124b0567479c875bb25" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 37483 } } peers { permanent_uuid: "aae8b08fd1ac4d2699d15a396e7ac8f2" member_type: VOTER last_known_addr { host: "127.4.159.189" port: 37399 } attrs { promote: false } } peers { permanent_uuid: "383bca56225440c4966e85359729cd9a" member_type: VOTER last_known_addr { host: "127.4.159.188" port: 35089 } attrs { promote: false } }
I20260807 08:48:04.829069 15699 sys_catalog.cc:455] T 00000000000000000000000000000000 P aae8b08fd1ac4d2699d15a396e7ac8f2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aae8b08fd1ac4d2699d15a396e7ac8f2. Latest consensus state: current_term: 2 leader_uuid: "aae8b08fd1ac4d2699d15a396e7ac8f2" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "82a928c4f0234124b0567479c875bb25" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 37483 } } peers { permanent_uuid: "aae8b08fd1ac4d2699d15a396e7ac8f2" member_type: VOTER last_known_addr { host: "127.4.159.189" port: 37399 } attrs { promote: false } } peers { permanent_uuid: "383bca56225440c4966e85359729cd9a" member_type: VOTER last_known_addr { host: "127.4.159.188" port: 35089 } attrs { promote: false } } }
I20260807 08:48:04.829159 15699 sys_catalog.cc:458] T 00000000000000000000000000000000 P aae8b08fd1ac4d2699d15a396e7ac8f2 [sys.catalog]: This master's current role is: LEADER
I20260807 08:48:04.829331 15701 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:48:04.829787 15701 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:48:04.829864 15701 catalog_manager.cc:1259] Loaded cluster ID: 6e5ed93ced5c45e9b23a567a01055589
I20260807 08:48:04.829895 15701 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:48:04.830090 15701 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:48:04.830163 15701 catalog_manager.cc:5879] T 00000000000000000000000000000000 P aae8b08fd1ac4d2699d15a396e7ac8f2: Loaded TSK: 0
I20260807 08:48:04.830293 15701 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:48:04.871138 14599 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49276:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 08:48:04.872699 14599 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:48:04.873615 15688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:04.873517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:37483 (local address 127.0.0.1:45414)
0807 08:48:04.873556 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:04.873584 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:37483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260807 08:48:04.873708 14900 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 383bca56225440c4966e85359729cd9a [term 2 FOLLOWER]: Refusing update from remote peer aae8b08fd1ac4d2699d15a396e7ac8f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260807 08:48:04.873946 15688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:04.873855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:37483 (local address 127.0.0.1:45418)
0807 08:48:04.873901 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:04.873929 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:37483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260807 08:48:04.873960 15683 consensus_queue.cc:1035] T 00000000000000000000000000000000 P aae8b08fd1ac4d2699d15a396e7ac8f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "383bca56225440c4966e85359729cd9a" member_type: VOTER last_known_addr { host: "127.4.159.188" port: 35089 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260807 08:48:04.874090 14569 consensus_peers.cc:487] T 00000000000000000000000000000000 P aae8b08fd1ac4d2699d15a396e7ac8f2 -> Peer 82a928c4f0234124b0567479c875bb25 (127.4.159.190:37483): Couldn't send request to peer 82a928c4f0234124b0567479c875bb25. Status: Network error: Client connection negotiation failed: client connection to 127.4.159.190:37483: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260807 08:48:04.874596 15686 sys_catalog.cc:455] T 00000000000000000000000000000000 P 383bca56225440c4966e85359729cd9a [sys.catalog]: SysCatalogTable state changed. Reason: New leader aae8b08fd1ac4d2699d15a396e7ac8f2. Latest consensus state: current_term: 2 leader_uuid: "aae8b08fd1ac4d2699d15a396e7ac8f2" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "82a928c4f0234124b0567479c875bb25" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 37483 } } peers { permanent_uuid: "aae8b08fd1ac4d2699d15a396e7ac8f2" member_type: VOTER last_known_addr { host: "127.4.159.189" port: 37399 } attrs { promote: false } } peers { permanent_uuid: "383bca56225440c4966e85359729cd9a" member_type: VOTER last_known_addr { host: "127.4.159.188" port: 35089 } attrs { promote: false } } }
I20260807 08:48:04.874686 15686 sys_catalog.cc:458] T 00000000000000000000000000000000 P 383bca56225440c4966e85359729cd9a [sys.catalog]: This master's current role is: FOLLOWER
I20260807 08:48:04.874965 15702 mvcc.cc:205] Tried to move back new op lower bound from 7315834818040606720 to 7315834817859014656. Current Snapshot: MvccSnapshot[applied={T|T < 7315834818040606720}]
I20260807 08:48:04.875046 15699 sys_catalog.cc:455] T 00000000000000000000000000000000 P aae8b08fd1ac4d2699d15a396e7ac8f2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "aae8b08fd1ac4d2699d15a396e7ac8f2" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "82a928c4f0234124b0567479c875bb25" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 37483 } } peers { permanent_uuid: "aae8b08fd1ac4d2699d15a396e7ac8f2" member_type: VOTER last_known_addr { host: "127.4.159.189" port: 37399 } attrs { promote: false } } peers { permanent_uuid: "383bca56225440c4966e85359729cd9a" member_type: VOTER last_known_addr { host: "127.4.159.188" port: 35089 } attrs { promote: false } } }
I20260807 08:48:04.875119 15699 sys_catalog.cc:458] T 00000000000000000000000000000000 P aae8b08fd1ac4d2699d15a396e7ac8f2 [sys.catalog]: This master's current role is: LEADER
I20260807 08:48:04.875181 15686 sys_catalog.cc:455] T 00000000000000000000000000000000 P 383bca56225440c4966e85359729cd9a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "aae8b08fd1ac4d2699d15a396e7ac8f2" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "82a928c4f0234124b0567479c875bb25" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 37483 } } peers { permanent_uuid: "aae8b08fd1ac4d2699d15a396e7ac8f2" member_type: VOTER last_known_addr { host: "127.4.159.189" port: 37399 } attrs { promote: false } } peers { permanent_uuid: "383bca56225440c4966e85359729cd9a" member_type: VOTER last_known_addr { host: "127.4.159.188" port: 35089 } attrs { promote: false } } }
I20260807 08:48:04.875279 15686 sys_catalog.cc:458] T 00000000000000000000000000000000 P 383bca56225440c4966e85359729cd9a [sys.catalog]: This master's current role is: FOLLOWER
I20260807 08:48:04.879910 15688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:04.876798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.131:43209 (local address 127.0.0.1:53514)
0807 08:48:04.876829 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:04.876839 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:04.876878 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:04.877133 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:04.877134 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:04.877135 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:48:04.877182 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:04.877184 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:04.877906 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:04.877907 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:04.878842 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:04.878845 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:04.879693 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:04.879694 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:04.879759 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:04.879764 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 08:48:04.879809 (+ 45us) client_negotiation.cc:240] Negotiation successful
0807 08:48:04.879848 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16}
I20260807 08:48:04.879992 15687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:04.876855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.129:35375 (local address 127.0.0.1:60120)
0807 08:48:04.876933 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:04.876941 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:04.876975 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:04.877317 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:04.877317 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:04.877318 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:48:04.877370 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:04.877371 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:04.878316 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:04.878317 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:04.879078 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:04.879080 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:04.879769 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:04.879769 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:04.879830 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:04.879832 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 08:48:04.879879 (+ 47us) client_negotiation.cc:240] Negotiation successful
0807 08:48:04.879912 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68,"mutex_wait_us":21}
I20260807 08:48:04.879997 15711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:04.877043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60120 (local address 127.4.159.129:35375)
0807 08:48:04.877175 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:04.877178 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:04.877186 (+ 8us) server_negotiation.cc:407] Connection header received
0807 08:48:04.877211 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:04.877212 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:04.877221 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:04.877429 (+ 208us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:48:04.877464 (+ 35us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:04.878212 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:04.879159 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:04.879704 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:04.879771 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:04.879782 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0807 08:48:04.879891 (+ 109us) server_negotiation.cc:299] Negotiation successful
0807 08:48:04.879924 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":50,"threads_started":1}
I20260807 08:48:04.880189 15710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:04.876864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53514 (local address 127.4.159.131:43209)
0807 08:48:04.877028 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:04.877030 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:04.877038 (+ 8us) server_negotiation.cc:407] Connection header received
0807 08:48:04.877064 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:04.877065 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:04.877076 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:04.877134 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:48:04.877260 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:04.877837 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:04.878937 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:04.879629 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:04.879687 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:04.879699 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0807 08:48:04.880074 (+ 375us) server_negotiation.cc:299] Negotiation successful
0807 08:48:04.880124 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":57,"threads_started":1}
I20260807 08:48:04.880249 15067 tablet_service.cc:1467] Processing CreateTablet for tablet 61a6c7c8f96b460bbd756aeefc06169a (DEFAULT_TABLE table=test-table [id=5745a6ae14164bc7b16ba6118c331e94]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:48:04.880409 15067 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 61a6c7c8f96b460bbd756aeefc06169a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:48:04.880752 15713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:04.877053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44234 (local address 127.4.159.130:39079)
0807 08:48:04.877178 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:04.877180 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:04.877187 (+ 7us) server_negotiation.cc:407] Connection header received
0807 08:48:04.877211 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:04.877212 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:04.877221 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:04.877428 (+ 207us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:48:04.877464 (+ 36us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:04.878216 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:04.879466 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:04.880398 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:04.880472 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:04.880486 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0807 08:48:04.880667 (+ 181us) server_negotiation.cc:299] Negotiation successful
0807 08:48:04.880700 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":66,"threads_started":1}
I20260807 08:48:04.880764 15698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:04.876939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.130:39079 (local address 127.0.0.1:44234)
0807 08:48:04.876988 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:04.876999 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:04.877050 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:04.877315 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:04.877316 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:04.877318 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:48:04.877370 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:04.877371 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:04.878293 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:04.878294 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:04.879391 (+ 1097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:04.879394 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:04.880507 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:04.880508 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:04.880579 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:04.880584 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 08:48:04.880660 (+ 76us) client_negotiation.cc:240] Negotiation successful
0807 08:48:04.880702 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22,"mutex_wait_us":26}
I20260807 08:48:04.881669 15716 tablet_bootstrap.cc:492] T 61a6c7c8f96b460bbd756aeefc06169a P 67262c924f4e4493bd8dcab7c92e6aaa: Bootstrap starting.
I20260807 08:48:04.881786 15488 tablet_service.cc:1467] Processing CreateTablet for tablet 61a6c7c8f96b460bbd756aeefc06169a (DEFAULT_TABLE table=test-table [id=5745a6ae14164bc7b16ba6118c331e94]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:48:04.881901 15488 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 61a6c7c8f96b460bbd756aeefc06169a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:48:04.882042 15716 tablet_bootstrap.cc:654] T 61a6c7c8f96b460bbd756aeefc06169a P 67262c924f4e4493bd8dcab7c92e6aaa: Neither blocks nor log segments found. Creating new log.
I20260807 08:48:04.882650 15716 tablet_bootstrap.cc:492] T 61a6c7c8f96b460bbd756aeefc06169a P 67262c924f4e4493bd8dcab7c92e6aaa: No bootstrap required, opened a new log
I20260807 08:48:04.882648 15279 tablet_service.cc:1467] Processing CreateTablet for tablet 61a6c7c8f96b460bbd756aeefc06169a (DEFAULT_TABLE table=test-table [id=5745a6ae14164bc7b16ba6118c331e94]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:48:04.882706 15716 ts_tablet_manager.cc:1397] T 61a6c7c8f96b460bbd756aeefc06169a P 67262c924f4e4493bd8dcab7c92e6aaa: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 08:48:04.882756 15279 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 61a6c7c8f96b460bbd756aeefc06169a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:48:04.882841 15716 raft_consensus.cc:348] T 61a6c7c8f96b460bbd756aeefc06169a P 67262c924f4e4493bd8dcab7c92e6aaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48c69edfbb324b37ace3f551898f2220" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 43209 } } peers { permanent_uuid: "67262c924f4e4493bd8dcab7c92e6aaa" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 35375 } } peers { permanent_uuid: "bed5f8e9f0ba46ea97101418a4f0affe" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 39079 } }
I20260807 08:48:04.882913 15716 raft_consensus.cc:374] T 61a6c7c8f96b460bbd756aeefc06169a P 67262c924f4e4493bd8dcab7c92e6aaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:48:04.882931 15716 raft_consensus.cc:729] T 61a6c7c8f96b460bbd756aeefc06169a P 67262c924f4e4493bd8dcab7c92e6aaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67262c924f4e4493bd8dcab7c92e6aaa, State: Initialized, Role: FOLLOWER
I20260807 08:48:04.882993 15716 consensus_queue.cc:260] T 61a6c7c8f96b460bbd756aeefc06169a P 67262c924f4e4493bd8dcab7c92e6aaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48c69edfbb324b37ace3f551898f2220" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 43209 } } peers { permanent_uuid: "67262c924f4e4493bd8dcab7c92e6aaa" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 35375 } } peers { permanent_uuid: "bed5f8e9f0ba46ea97101418a4f0affe" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 39079 } }
I20260807 08:48:04.883126 15716 ts_tablet_manager.cc:1428] T 61a6c7c8f96b460bbd756aeefc06169a P 67262c924f4e4493bd8dcab7c92e6aaa: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260807 08:48:04.883236 15721 tablet_bootstrap.cc:492] T 61a6c7c8f96b460bbd756aeefc06169a P 48c69edfbb324b37ace3f551898f2220: Bootstrap starting.
I20260807 08:48:04.883589 15721 tablet_bootstrap.cc:654] T 61a6c7c8f96b460bbd756aeefc06169a P 48c69edfbb324b37ace3f551898f2220: Neither blocks nor log segments found. Creating new log.
I20260807 08:48:04.884130 15723 tablet_bootstrap.cc:492] T 61a6c7c8f96b460bbd756aeefc06169a P bed5f8e9f0ba46ea97101418a4f0affe: Bootstrap starting.
I20260807 08:48:04.884307 15721 tablet_bootstrap.cc:492] T 61a6c7c8f96b460bbd756aeefc06169a P 48c69edfbb324b37ace3f551898f2220: No bootstrap required, opened a new log
I20260807 08:48:04.884357 15721 ts_tablet_manager.cc:1397] T 61a6c7c8f96b460bbd756aeefc06169a P 48c69edfbb324b37ace3f551898f2220: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 08:48:04.884475 15721 raft_consensus.cc:348] T 61a6c7c8f96b460bbd756aeefc06169a P 48c69edfbb324b37ace3f551898f2220 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48c69edfbb324b37ace3f551898f2220" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 43209 } } peers { permanent_uuid: "67262c924f4e4493bd8dcab7c92e6aaa" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 35375 } } peers { permanent_uuid: "bed5f8e9f0ba46ea97101418a4f0affe" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 39079 } }
I20260807 08:48:04.884539 15721 raft_consensus.cc:374] T 61a6c7c8f96b460bbd756aeefc06169a P 48c69edfbb324b37ace3f551898f2220 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:48:04.884558 15721 raft_consensus.cc:729] T 61a6c7c8f96b460bbd756aeefc06169a P 48c69edfbb324b37ace3f551898f2220 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48c69edfbb324b37ace3f551898f2220, State: Initialized, Role: FOLLOWER
I20260807 08:48:04.884613 15721 consensus_queue.cc:260] T 61a6c7c8f96b460bbd756aeefc06169a P 48c69edfbb324b37ace3f551898f2220 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48c69edfbb324b37ace3f551898f2220" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 43209 } } peers { permanent_uuid: "67262c924f4e4493bd8dcab7c92e6aaa" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 35375 } } peers { permanent_uuid: "bed5f8e9f0ba46ea97101418a4f0affe" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 39079 } }
I20260807 08:48:04.884618 15723 tablet_bootstrap.cc:654] T 61a6c7c8f96b460bbd756aeefc06169a P bed5f8e9f0ba46ea97101418a4f0affe: Neither blocks nor log segments found. Creating new log.
I20260807 08:48:04.884768 15721 ts_tablet_manager.cc:1428] T 61a6c7c8f96b460bbd756aeefc06169a P 48c69edfbb324b37ace3f551898f2220: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 08:48:04.885164 15732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:04.884908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:37483 (local address 127.4.159.131:45693)
0807 08:48:04.885096 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:04.885132 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:37483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":76,"threads_started":1}
I20260807 08:48:04.885188 15723 tablet_bootstrap.cc:492] T 61a6c7c8f96b460bbd756aeefc06169a P bed5f8e9f0ba46ea97101418a4f0affe: No bootstrap required, opened a new log
I20260807 08:48:04.885231 15723 ts_tablet_manager.cc:1397] T 61a6c7c8f96b460bbd756aeefc06169a P bed5f8e9f0ba46ea97101418a4f0affe: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 08:48:04.885236 15586 heartbeater.cc:503] Master 127.4.159.189:37399 was elected leader, sending a full tablet report...
I20260807 08:48:04.885361 15723 raft_consensus.cc:348] T 61a6c7c8f96b460bbd756aeefc06169a P bed5f8e9f0ba46ea97101418a4f0affe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48c69edfbb324b37ace3f551898f2220" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 43209 } } peers { permanent_uuid: "67262c924f4e4493bd8dcab7c92e6aaa" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 35375 } } peers { permanent_uuid: "bed5f8e9f0ba46ea97101418a4f0affe" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 39079 } }
I20260807 08:48:04.885437 15723 raft_consensus.cc:374] T 61a6c7c8f96b460bbd756aeefc06169a P bed5f8e9f0ba46ea97101418a4f0affe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:48:04.885461 15723 raft_consensus.cc:729] T 61a6c7c8f96b460bbd756aeefc06169a P bed5f8e9f0ba46ea97101418a4f0affe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bed5f8e9f0ba46ea97101418a4f0affe, State: Initialized, Role: FOLLOWER
I20260807 08:48:04.885499 15723 consensus_queue.cc:260] T 61a6c7c8f96b460bbd756aeefc06169a P bed5f8e9f0ba46ea97101418a4f0affe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48c69edfbb324b37ace3f551898f2220" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 43209 } } peers { permanent_uuid: "67262c924f4e4493bd8dcab7c92e6aaa" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 35375 } } peers { permanent_uuid: "bed5f8e9f0ba46ea97101418a4f0affe" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 39079 } }
I20260807 08:48:04.885648 15732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:04.885490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:37483 (local address 127.4.159.131:48911)
0807 08:48:04.885535 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:04.885609 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:37483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260807 08:48:04.885654 15723 ts_tablet_manager.cc:1428] T 61a6c7c8f96b460bbd756aeefc06169a P bed5f8e9f0ba46ea97101418a4f0affe: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260807 08:48:04.885958 15731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:04.884822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:37483 (local address 127.4.159.129:47757)
0807 08:48:04.885888 (+ 1066us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:04.885924 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:37483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1044,"thread_start_us":81,"threads_started":1}
I20260807 08:48:04.886114 15738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:04.885861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:37483 (local address 127.4.159.130:41537)
0807 08:48:04.885977 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:04.886079 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:37483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":80,"threads_started":1}
I20260807 08:48:04.886171 15392 heartbeater.cc:503] Master 127.4.159.189:37399 was elected leader, sending a full tablet report...
I20260807 08:48:04.886585 15738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:04.886443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:37483 (local address 127.4.159.130:59711)
0807 08:48:04.886533 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:04.886562 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:37483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260807 08:48:05.036115 15733 raft_consensus.cc:482] T 61a6c7c8f96b460bbd756aeefc06169a P bed5f8e9f0ba46ea97101418a4f0affe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:48:05.036201 15733 raft_consensus.cc:504] T 61a6c7c8f96b460bbd756aeefc06169a P bed5f8e9f0ba46ea97101418a4f0affe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48c69edfbb324b37ace3f551898f2220" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 43209 } } peers { permanent_uuid: "67262c924f4e4493bd8dcab7c92e6aaa" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 35375 } } peers { permanent_uuid: "bed5f8e9f0ba46ea97101418a4f0affe" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 39079 } }
I20260807 08:48:05.036516 15733 leader_election.cc:288] T 61a6c7c8f96b460bbd756aeefc06169a P bed5f8e9f0ba46ea97101418a4f0affe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48c69edfbb324b37ace3f551898f2220 (127.4.159.131:43209), 67262c924f4e4493bd8dcab7c92e6aaa (127.4.159.129:35375)
I20260807 08:48:05.039194 15710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:05.036736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.130:54989 (local address 127.4.159.131:43209)
0807 08:48:05.036787 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:05.036789 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:05.036888 (+ 99us) server_negotiation.cc:407] Connection header received
0807 08:48:05.036938 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:05.036938 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:05.036949 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:05.036996 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:48:05.037130 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:05.037609 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:05.038560 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:05.038912 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:05.038974 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:05.038985 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0807 08:48:05.039100 (+ 115us) server_negotiation.cc:299] Negotiation successful
0807 08:48:05.039132 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260807 08:48:05.039232 15738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:05.036663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.131:43209 (local address 127.4.159.130:54989)
0807 08:48:05.036805 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:05.036818 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:05.036887 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:05.037012 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:05.037013 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:05.037015 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:48:05.037072 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:05.037074 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:05.037685 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:05.037686 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:05.038485 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:05.038487 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:05.038981 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:05.038982 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:05.039042 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:05.039045 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 08:48:05.039089 (+ 44us) client_negotiation.cc:240] Negotiation successful
0807 08:48:05.039127 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122}
I20260807 08:48:05.039235 15711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:05.036739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.130:48993 (local address 127.4.159.129:35375)
0807 08:48:05.036784 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:05.036786 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:05.036927 (+ 141us) server_negotiation.cc:407] Connection header received
0807 08:48:05.036978 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:05.036979 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:05.036988 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:05.037034 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:48:05.037181 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:05.037658 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:05.038575 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:05.038926 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:05.038986 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:05.038996 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0807 08:48:05.039129 (+ 133us) server_negotiation.cc:299] Negotiation successful
0807 08:48:05.039172 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 08:48:05.039255 15739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:05.036665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.129:35375 (local address 127.4.159.130:48993)
0807 08:48:05.036860 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:05.036870 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:05.036927 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:05.037041 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:05.037042 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:05.037044 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:48:05.037115 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:05.037116 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:05.037724 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:05.037726 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:05.038499 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:05.038502 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:05.039012 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:05.039013 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:05.039070 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:05.039072 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 08:48:05.039116 (+ 44us) client_negotiation.cc:240] Negotiation successful
0807 08:48:05.039154 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"mutex_wait_us":38,"thread_start_us":106,"threads_started":1}
I20260807 08:48:05.039454 15518 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61a6c7c8f96b460bbd756aeefc06169a" candidate_uuid: "bed5f8e9f0ba46ea97101418a4f0affe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48c69edfbb324b37ace3f551898f2220" is_pre_election: true
I20260807 08:48:05.039542 15518 raft_consensus.cc:2454] T 61a6c7c8f96b460bbd756aeefc06169a P 48c69edfbb324b37ace3f551898f2220 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bed5f8e9f0ba46ea97101418a4f0affe in term 0.
I20260807 08:48:05.039546 15099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61a6c7c8f96b460bbd756aeefc06169a" candidate_uuid: "bed5f8e9f0ba46ea97101418a4f0affe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67262c924f4e4493bd8dcab7c92e6aaa" is_pre_election: true
I20260807 08:48:05.039686 15099 raft_consensus.cc:2454] T 61a6c7c8f96b460bbd756aeefc06169a P 67262c924f4e4493bd8dcab7c92e6aaa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bed5f8e9f0ba46ea97101418a4f0affe in term 0.
I20260807 08:48:05.039695 15208 leader_election.cc:302] T 61a6c7c8f96b460bbd756aeefc06169a P bed5f8e9f0ba46ea97101418a4f0affe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48c69edfbb324b37ace3f551898f2220, bed5f8e9f0ba46ea97101418a4f0affe; no voters:
I20260807 08:48:05.039847 15733 raft_consensus.cc:2781] T 61a6c7c8f96b460bbd756aeefc06169a P bed5f8e9f0ba46ea97101418a4f0affe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:48:05.039891 15733 raft_consensus.cc:482] T 61a6c7c8f96b460bbd756aeefc06169a P bed5f8e9f0ba46ea97101418a4f0affe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:48:05.039909 15733 raft_consensus.cc:3037] T 61a6c7c8f96b460bbd756aeefc06169a P bed5f8e9f0ba46ea97101418a4f0affe [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:48:05.040562 15733 raft_consensus.cc:504] T 61a6c7c8f96b460bbd756aeefc06169a P bed5f8e9f0ba46ea97101418a4f0affe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48c69edfbb324b37ace3f551898f2220" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 43209 } } peers { permanent_uuid: "67262c924f4e4493bd8dcab7c92e6aaa" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 35375 } } peers { permanent_uuid: "bed5f8e9f0ba46ea97101418a4f0affe" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 39079 } }
I20260807 08:48:05.040675 15733 leader_election.cc:288] T 61a6c7c8f96b460bbd756aeefc06169a P bed5f8e9f0ba46ea97101418a4f0affe [CANDIDATE]: Term 1 election: Requested vote from peers 48c69edfbb324b37ace3f551898f2220 (127.4.159.131:43209), 67262c924f4e4493bd8dcab7c92e6aaa (127.4.159.129:35375)
I20260807 08:48:05.040822 15518 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61a6c7c8f96b460bbd756aeefc06169a" candidate_uuid: "bed5f8e9f0ba46ea97101418a4f0affe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48c69edfbb324b37ace3f551898f2220"
I20260807 08:48:05.040904 15518 raft_consensus.cc:3037] T 61a6c7c8f96b460bbd756aeefc06169a P 48c69edfbb324b37ace3f551898f2220 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:48:05.040910 15099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61a6c7c8f96b460bbd756aeefc06169a" candidate_uuid: "bed5f8e9f0ba46ea97101418a4f0affe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67262c924f4e4493bd8dcab7c92e6aaa"
I20260807 08:48:05.040979 15099 raft_consensus.cc:3037] T 61a6c7c8f96b460bbd756aeefc06169a P 67262c924f4e4493bd8dcab7c92e6aaa [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:48:05.041483 15518 raft_consensus.cc:2454] T 61a6c7c8f96b460bbd756aeefc06169a P 48c69edfbb324b37ace3f551898f2220 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bed5f8e9f0ba46ea97101418a4f0affe in term 1.
I20260807 08:48:05.041611 15099 raft_consensus.cc:2454] T 61a6c7c8f96b460bbd756aeefc06169a P 67262c924f4e4493bd8dcab7c92e6aaa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bed5f8e9f0ba46ea97101418a4f0affe in term 1.
I20260807 08:48:05.041639 15208 leader_election.cc:302] T 61a6c7c8f96b460bbd756aeefc06169a P bed5f8e9f0ba46ea97101418a4f0affe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48c69edfbb324b37ace3f551898f2220, bed5f8e9f0ba46ea97101418a4f0affe; no voters:
I20260807 08:48:05.041710 15733 raft_consensus.cc:2781] T 61a6c7c8f96b460bbd756aeefc06169a P bed5f8e9f0ba46ea97101418a4f0affe [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:48:05.041844 15733 raft_consensus.cc:686] T 61a6c7c8f96b460bbd756aeefc06169a P bed5f8e9f0ba46ea97101418a4f0affe [term 1 LEADER]: Becoming Leader. State: Replica: bed5f8e9f0ba46ea97101418a4f0affe, State: Running, Role: LEADER
I20260807 08:48:05.041932 15733 consensus_queue.cc:237] T 61a6c7c8f96b460bbd756aeefc06169a P bed5f8e9f0ba46ea97101418a4f0affe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48c69edfbb324b37ace3f551898f2220" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 43209 } } peers { permanent_uuid: "67262c924f4e4493bd8dcab7c92e6aaa" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 35375 } } peers { permanent_uuid: "bed5f8e9f0ba46ea97101418a4f0affe" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 39079 } }
I20260807 08:48:05.042094 15738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:05.042012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:37483 (local address 127.4.159.130:42087)
0807 08:48:05.042047 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:05.042069 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:37483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260807 08:48:05.042586 14599 catalog_manager.cc:5495] T 61a6c7c8f96b460bbd756aeefc06169a P bed5f8e9f0ba46ea97101418a4f0affe reported cstate change: term changed from 0 to 1, leader changed from <none> to bed5f8e9f0ba46ea97101418a4f0affe (127.4.159.130). New cstate: current_term: 1 leader_uuid: "bed5f8e9f0ba46ea97101418a4f0affe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48c69edfbb324b37ace3f551898f2220" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 43209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "67262c924f4e4493bd8dcab7c92e6aaa" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 35375 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bed5f8e9f0ba46ea97101418a4f0affe" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 39079 } health_report { overall_health: HEALTHY } } }
I20260807 08:48:05.057945 15713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:05.055909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44240 (local address 127.4.159.130:39079)
0807 08:48:05.055952 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:05.055954 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:05.055962 (+ 8us) server_negotiation.cc:407] Connection header received
0807 08:48:05.056020 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:05.056020 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:05.056038 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:05.056094 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:48:05.056208 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:05.056619 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:05.057313 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:05.057486 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:05.057538 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:05.057683 (+ 145us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:48:05.057770 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:48:05.057829 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 08:48:05.057868 (+ 39us) server_negotiation.cc:299] Negotiation successful
0807 08:48:05.057886 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 08:48:05.058801 15518 raft_consensus.cc:1261] T 61a6c7c8f96b460bbd756aeefc06169a P 48c69edfbb324b37ace3f551898f2220 [term 1 FOLLOWER]: Refusing update from remote peer bed5f8e9f0ba46ea97101418a4f0affe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:48:05.058910 15099 raft_consensus.cc:1261] T 61a6c7c8f96b460bbd756aeefc06169a P 67262c924f4e4493bd8dcab7c92e6aaa [term 1 FOLLOWER]: Refusing update from remote peer bed5f8e9f0ba46ea97101418a4f0affe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:48:05.059042 15733 consensus_queue.cc:1035] T 61a6c7c8f96b460bbd756aeefc06169a P bed5f8e9f0ba46ea97101418a4f0affe [LEADER]: Connected to new peer: Peer: permanent_uuid: "48c69edfbb324b37ace3f551898f2220" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 43209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:48:05.059170 15741 consensus_queue.cc:1035] T 61a6c7c8f96b460bbd756aeefc06169a P bed5f8e9f0ba46ea97101418a4f0affe [LEADER]: Connected to new peer: Peer: permanent_uuid: "67262c924f4e4493bd8dcab7c92e6aaa" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 35375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:48:05.059681 15170 heartbeater.cc:503] Master 127.4.159.189:37399 was elected leader, sending a full tablet report...
I20260807 08:48:05.060408 15732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:05.060314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:37483 (local address 127.4.159.131:53399)
0807 08:48:05.060351 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:05.060379 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:37483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260807 08:48:05.060992 15738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:05.060910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:37483 (local address 127.4.159.130:38595)
0807 08:48:05.060944 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:05.060967 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:37483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260807 08:48:05.061436 15732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:05.061341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:37483 (local address 127.4.159.131:40523)
0807 08:48:05.061389 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:05.061412 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:37483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260807 08:48:05.061913 15731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:05.059902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:37483 (local address 127.4.159.129:59399)
0807 08:48:05.061857 (+ 1955us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:05.061886 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:37483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1940}
I20260807 08:48:05.063061 15731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:05.062772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:37483 (local address 127.4.159.129:60969)
0807 08:48:05.063005 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:05.063035 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:37483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214}
I20260807 08:48:05.063886 14599 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49276:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:48:05.063962 14599 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49276:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:48:05.065042 14599 catalog_manager.cc:5782] T 00000000000000000000000000000000 P aae8b08fd1ac4d2699d15a396e7ac8f2: Sending DeleteTablet for 3 replicas of tablet 61a6c7c8f96b460bbd756aeefc06169a
I20260807 08:48:05.065307 15488 tablet_service.cc:1514] Processing DeleteTablet for tablet 61a6c7c8f96b460bbd756aeefc06169a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:48:05 UTC) from {username='oryx', principal='oryx/127.4.159.189@KRBTEST.COM'} at 127.0.0.1:53514
I20260807 08:48:05.065455 15279 tablet_service.cc:1514] Processing DeleteTablet for tablet 61a6c7c8f96b460bbd756aeefc06169a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:48:05 UTC) from {username='oryx', principal='oryx/127.4.159.189@KRBTEST.COM'} at 127.0.0.1:44234
I20260807 08:48:05.065536 15067 tablet_service.cc:1514] Processing DeleteTablet for tablet 61a6c7c8f96b460bbd756aeefc06169a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:48:05 UTC) from {username='oryx', principal='oryx/127.4.159.189@KRBTEST.COM'} at 127.0.0.1:60120
I20260807 08:48:05.065701 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 14970
I20260807 08:48:05.065722 15770 tablet_replica.cc:330] T 61a6c7c8f96b460bbd756aeefc06169a P 67262c924f4e4493bd8dcab7c92e6aaa: stopping tablet replica
I20260807 08:48:05.065723 15766 tablet_replica.cc:330] T 61a6c7c8f96b460bbd756aeefc06169a P 48c69edfbb324b37ace3f551898f2220: stopping tablet replica
I20260807 08:48:05.065768 15770 raft_consensus.cc:2229] T 61a6c7c8f96b460bbd756aeefc06169a P 67262c924f4e4493bd8dcab7c92e6aaa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:48:05.065771 15766 raft_consensus.cc:2229] T 61a6c7c8f96b460bbd756aeefc06169a P 48c69edfbb324b37ace3f551898f2220 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:48:05.065805 15766 raft_consensus.cc:2258] T 61a6c7c8f96b460bbd756aeefc06169a P 48c69edfbb324b37ace3f551898f2220 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:48:05.065802 15770 raft_consensus.cc:2258] T 61a6c7c8f96b460bbd756aeefc06169a P 67262c924f4e4493bd8dcab7c92e6aaa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:48:05.070001 15766 ts_tablet_manager.cc:1905] T 61a6c7c8f96b460bbd756aeefc06169a P 48c69edfbb324b37ace3f551898f2220: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:48:05.071494 15766 ts_tablet_manager.cc:1918] T 61a6c7c8f96b460bbd756aeefc06169a P 48c69edfbb324b37ace3f551898f2220: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:48:05.071543 15766 log.cc:1196] T 61a6c7c8f96b460bbd756aeefc06169a P 48c69edfbb324b37ace3f551898f2220: Deleting WAL directory at /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786092453486433-4734-0/minicluster-data/ts-2/wal/wals/61a6c7c8f96b460bbd756aeefc06169a
I20260807 08:48:05.071764 15766 ts_tablet_manager.cc:1939] T 61a6c7c8f96b460bbd756aeefc06169a P 48c69edfbb324b37ace3f551898f2220: Deleting consensus metadata
I20260807 08:48:05.072124 14569 catalog_manager.cc:4841] TS 48c69edfbb324b37ace3f551898f2220 (127.4.159.131:43209): tablet 61a6c7c8f96b460bbd756aeefc06169a (table test-table [id=5745a6ae14164bc7b16ba6118c331e94]) successfully deleted
I20260807 08:48:05.074550 15769 tablet_replica.cc:330] T 61a6c7c8f96b460bbd756aeefc06169a P bed5f8e9f0ba46ea97101418a4f0affe: stopping tablet replica
I20260807 08:48:05.074613 15769 raft_consensus.cc:2229] T 61a6c7c8f96b460bbd756aeefc06169a P bed5f8e9f0ba46ea97101418a4f0affe [term 1 LEADER]: Raft consensus shutting down.
I20260807 08:48:05.074694 15769 raft_consensus.cc:2258] T 61a6c7c8f96b460bbd756aeefc06169a P bed5f8e9f0ba46ea97101418a4f0affe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:48:05.075136 15769 ts_tablet_manager.cc:1905] T 61a6c7c8f96b460bbd756aeefc06169a P bed5f8e9f0ba46ea97101418a4f0affe: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260807 08:48:05.075507 15698 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:48:05.075396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.129:35375 (local address 127.0.0.1:60128)
0807 08:48:05.075439 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:05.075463 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.129:35375: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":27}
W20260807 08:48:05.075626 14572 catalog_manager.cc:4584] TS 67262c924f4e4493bd8dcab7c92e6aaa (127.4.159.129:35375): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 61a6c7c8f96b460bbd756aeefc06169a: Network error: Client connection negotiation failed: client connection to 127.4.159.129:35375: connect: Connection reset by peer (error 104)
I20260807 08:48:05.075752 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 15176
I20260807 08:48:05.084793 15698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:05.084697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.130:39079 (local address 127.0.0.1:44248)
0807 08:48:05.084742 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:05.084767 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.130:39079: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260807 08:48:05.084803 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 15397
I20260807 08:48:05.092806 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 14377
I20260807 08:48:05.100785 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 14677
2026-08-07T08:48:05Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (6415 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786092453486433-4734-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:48:05 dist-test-slave-6zbq krb5kdc[15775](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:48:05 dist-test-slave-6zbq krb5kdc[15775](info): set up 2 sockets
Aug 07 08:48:05 dist-test-slave-6zbq krb5kdc[15775](info): commencing operation
krb5kdc: starting...
W20260807 08:48:06.218619 4734 mini_kdc.cc:121] Time spent starting KDC: real 1.094s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786092453486433-4734-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786092453486433-4734-0/krb5kdc/test-user.keytab.
Aug 07 08:48:06 dist-test-slave-6zbq krb5kdc[15775](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092486, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:48:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:48:06Z Disabled control of system clock
WARNING: no policy specified for oryx/127.4.159.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.159.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.159.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786092453486433-4734-0/krb5kdc/oryx_127.4.159.190.keytab.
Entry for principal oryx/127.4.159.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786092453486433-4734-0/krb5kdc/oryx_127.4.159.190.keytab.
I20260807 08:48:06.330572 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786092453486433-4734-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786092453486433-4734-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786092453486433-4734-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.159.190:45303
--webserver_interface=127.4.159.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786092453486433-4734-0/krb5kdc/oryx_127.4.159.190.keytab
--principal=oryx/127.4.159.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:45103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.159.190:45303
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:48:06.408946 15797 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:48:06.409121 15797 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:48:06.409142 15797 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:48:06.410250 15797 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:48:06.410288 15797 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 08:48:06.410300 15797 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:48:06.410324 15797 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:48:06.410364 15797 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:48:06.410387 15797 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:48:06.411826 15797 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:45103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786092453486433-4734-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786092453486433-4734-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.159.190:45303
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786092453486433-4734-0/krb5kdc/oryx_127.4.159.190.keytab
--principal=oryx/127.4.159.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.159.190:45303
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.159.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15797
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786092453486433-4734-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:48:06.412036 15797 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:48:06.412227 15797 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:48:06.416607 15805 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:48:06.416615 15809 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:48:06.416723 15797 server_base.cc:1034] running on GCE node
W20260807 08:48:06.416615 15804 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:48:06.417100 15797 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:48:06.417372 15797 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:48:06.418494 15797 hybrid_clock.cc:648] HybridClock initialized: now 1786092486418486 us; error 26 us; skew 500 ppm
Aug 07 08:48:06 dist-test-slave-6zbq krb5kdc[15775](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092486, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.159.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:48:06.420764 15797 init.cc:377] Logged in from keytab as oryx/127.4.159.190@KRBTEST.COM (short username oryx)
I20260807 08:48:06.421190 15797 webserver.cc:460] Webserver started at http://127.4.159.190:33889/ using document root <none> and password file <none>
I20260807 08:48:06.421377 15797 fs_manager.cc:359] Metadata directory not provided
I20260807 08:48:06.421423 15797 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:48:06.421540 15797 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:48:06.422554 15797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786092453486433-4734-0/minicluster-data/master-0/data/instance:
uuid: "4ea7dd9dc02e498b932350ceaf4214cb"
format_stamp: "Formatted at 2026-08-07 08:48:06 on dist-test-slave-6zbq"
server_key: "ef466b8c8c2ddc2be5500012311575bc"
server_key_iv: "0a814ba2de1a90844abcd5e40fa4b524"
server_key_version: "encryptionkey@0"
I20260807 08:48:06.422878 15797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786092453486433-4734-0/minicluster-data/master-0/wal/instance:
uuid: "4ea7dd9dc02e498b932350ceaf4214cb"
format_stamp: "Formatted at 2026-08-07 08:48:06 on dist-test-slave-6zbq"
server_key: "ef466b8c8c2ddc2be5500012311575bc"
server_key_iv: "0a814ba2de1a90844abcd5e40fa4b524"
server_key_version: "encryptionkey@0"
I20260807 08:48:06.424088 15797 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 08:48:06.424731 15821 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:48:06.424888 15797 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 08:48:06.424968 15797 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786092453486433-4734-0/minicluster-data/master-0/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786092453486433-4734-0/minicluster-data/master-0/wal
uuid: "4ea7dd9dc02e498b932350ceaf4214cb"
format_stamp: "Formatted at 2026-08-07 08:48:06 on dist-test-slave-6zbq"
server_key: "ef466b8c8c2ddc2be5500012311575bc"
server_key_iv: "0a814ba2de1a90844abcd5e40fa4b524"
server_key_version: "encryptionkey@0"
I20260807 08:48:06.425019 15797 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786092453486433-4734-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786092453486433-4734-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786092453486433-4734-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:48:06.440116 15797 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:48:06.441690 15797 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:48:06.441792 15797 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:48:06.445960 15797 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.190:45303
I20260807 08:48:06.446025 15922 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.190:45303 every 8 connection(s)
I20260807 08:48:06.446305 15797 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
I20260807 08:48:06.446969 15924 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:48:06.449074 15924 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ea7dd9dc02e498b932350ceaf4214cb: Bootstrap starting.
I20260807 08:48:06.449640 15924 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4ea7dd9dc02e498b932350ceaf4214cb: Neither blocks nor log segments found. Creating new log.
I20260807 08:48:06.449899 15924 log.cc:824] T 00000000000000000000000000000000 P 4ea7dd9dc02e498b932350ceaf4214cb: Log is configured to *not* fsync() on all Append() calls
I20260807 08:48:06.450538 15924 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ea7dd9dc02e498b932350ceaf4214cb: No bootstrap required, opened a new log
I20260807 08:48:06.452531 15924 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4ea7dd9dc02e498b932350ceaf4214cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ea7dd9dc02e498b932350ceaf4214cb" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 45303 } }
I20260807 08:48:06.452664 15924 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4ea7dd9dc02e498b932350ceaf4214cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:48:06.452696 15924 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4ea7dd9dc02e498b932350ceaf4214cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ea7dd9dc02e498b932350ceaf4214cb, State: Initialized, Role: FOLLOWER
I20260807 08:48:06.452821 15924 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4ea7dd9dc02e498b932350ceaf4214cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ea7dd9dc02e498b932350ceaf4214cb" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 45303 } }
I20260807 08:48:06.452880 15924 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4ea7dd9dc02e498b932350ceaf4214cb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:48:06.452909 15924 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4ea7dd9dc02e498b932350ceaf4214cb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:48:06.452956 15924 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4ea7dd9dc02e498b932350ceaf4214cb [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:48:06.453449 15924 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4ea7dd9dc02e498b932350ceaf4214cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ea7dd9dc02e498b932350ceaf4214cb" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 45303 } }
I20260807 08:48:06.453539 15924 leader_election.cc:302] T 00000000000000000000000000000000 P 4ea7dd9dc02e498b932350ceaf4214cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4ea7dd9dc02e498b932350ceaf4214cb; no voters:
I20260807 08:48:06.453685 15924 leader_election.cc:288] T 00000000000000000000000000000000 P 4ea7dd9dc02e498b932350ceaf4214cb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:48:06.453732 15928 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4ea7dd9dc02e498b932350ceaf4214cb [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:48:06.453879 15928 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4ea7dd9dc02e498b932350ceaf4214cb [term 1 LEADER]: Becoming Leader. State: Replica: 4ea7dd9dc02e498b932350ceaf4214cb, State: Running, Role: LEADER
I20260807 08:48:06.453912 15924 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4ea7dd9dc02e498b932350ceaf4214cb [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:48:06.453979 15928 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4ea7dd9dc02e498b932350ceaf4214cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ea7dd9dc02e498b932350ceaf4214cb" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 45303 } }
I20260807 08:48:06.454316 15933 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ea7dd9dc02e498b932350ceaf4214cb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4ea7dd9dc02e498b932350ceaf4214cb. Latest consensus state: current_term: 1 leader_uuid: "4ea7dd9dc02e498b932350ceaf4214cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ea7dd9dc02e498b932350ceaf4214cb" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 45303 } } }
I20260807 08:48:06.454429 15933 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ea7dd9dc02e498b932350ceaf4214cb [sys.catalog]: This master's current role is: LEADER
I20260807 08:48:06.454298 15932 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ea7dd9dc02e498b932350ceaf4214cb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4ea7dd9dc02e498b932350ceaf4214cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ea7dd9dc02e498b932350ceaf4214cb" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 45303 } } }
I20260807 08:48:06.454485 15932 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ea7dd9dc02e498b932350ceaf4214cb [sys.catalog]: This master's current role is: LEADER
I20260807 08:48:06.454607 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 15797
I20260807 08:48:06.454630 15942 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:48:06.454704 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786092453486433-4734-0/minicluster-data/master-0/wal/instance
I20260807 08:48:06.454903 4734 external_mini_cluster.cc:1428] Setting key c56c41a6a607f601cf7a2a381b3f5f96
I20260807 08:48:06.455200 15942 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:48:06.457235 15942 catalog_manager.cc:1347] Generated new cluster ID: 82aea9559f9f41ebb5febd9844625eb4
I20260807 08:48:06.457283 15942 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 08:48:06 dist-test-slave-6zbq krb5kdc[15775](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092486, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM
I20260807 08:48:06.466501 15948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:06.455328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52318 (local address 127.4.159.190:45303)
0807 08:48:06.456351 (+ 1023us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:06.456360 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:06.456382 (+ 22us) server_negotiation.cc:407] Connection header received
0807 08:48:06.456466 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:06.456471 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:06.456510 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:06.456608 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:06.457412 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:06.458781 (+ 1369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:06.459816 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:06.460382 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:06.460473 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:06.463079 (+ 2606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:48:06.463084 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:48:06.463085 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:48:06.463097 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:48:06.464867 (+ 1770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:48:06.465216 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:48:06.465219 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:48:06.465219 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:48:06.465323 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:48:06.465495 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:48:06.465496 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:48:06.465496 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:48:06.465780 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:48:06.465929 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0807 08:48:06.465993 (+ 64us) server_negotiation.cc:299] Negotiation successful
0807 08:48:06.466251 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":982,"thread_start_us":73,"threads_started":1}
I20260807 08:48:06.467080 15942 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:48:06.467527 15942 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:48:06.471840 15942 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4ea7dd9dc02e498b932350ceaf4214cb: Generated new TSK 0
I20260807 08:48:06.471983 15942 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.4.159.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.159.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.159.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786092453486433-4734-0/krb5kdc/oryx_127.4.159.129.keytab.
Entry for principal oryx/127.4.159.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786092453486433-4734-0/krb5kdc/oryx_127.4.159.129.keytab.
I20260807 08:48:06.501916 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786092453486433-4734-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786092453486433-4734-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786092453486433-4734-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786092453486433-4734-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.159.129:0
--local_ip_for_outbound_sockets=127.4.159.129
--webserver_interface=127.4.159.129
--webserver_port=0
--tserver_master_addrs=127.4.159.190:45303
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786092453486433-4734-0/krb5kdc/oryx_127.4.159.129.keytab
--principal=oryx/127.4.159.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:45103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:48:06.579550 15966 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:48:06.579730 15966 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:48:06.579752 15966 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:48:06.580835 15966 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:48:06.580873 15966 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:48:06.580894 15966 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 08:48:06.580924 15966 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.159.129
I20260807 08:48:06.582388 15966 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:45103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786092453486433-4734-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786092453486433-4734-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786092453486433-4734-0/krb5kdc/oryx_127.4.159.129.keytab
--principal=oryx/127.4.159.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.159.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786092453486433-4734-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.159.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.4.159.190:45303
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15966
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.159.129
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786092453486433-4734-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:48:06.582582 15966 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:48:06.582779 15966 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:48:06.587252 15972 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:48:06.587258 15973 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:48:06.587321 15966 server_base.cc:1034] running on GCE node
W20260807 08:48:06.587270 15976 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:48:06.587589 15966 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:48:06.587823 15966 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:48:06.588965 15966 hybrid_clock.cc:648] HybridClock initialized: now 1786092486588954 us; error 29 us; skew 500 ppm
Aug 07 08:48:06 dist-test-slave-6zbq krb5kdc[15775](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092486, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.159.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:48:06.591276 15966 init.cc:377] Logged in from keytab as oryx/127.4.159.129@KRBTEST.COM (short username oryx)
I20260807 08:48:06.591634 15966 webserver.cc:460] Webserver started at http://127.4.159.129:36601/ using document root <none> and password file <none>
I20260807 08:48:06.591799 15966 fs_manager.cc:359] Metadata directory not provided
I20260807 08:48:06.591835 15966 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:48:06.591923 15966 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:48:06.592844 15966 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786092453486433-4734-0/minicluster-data/ts-0/data/instance:
uuid: "4f8ad558036e4bcbbace591712a92519"
format_stamp: "Formatted at 2026-08-07 08:48:06 on dist-test-slave-6zbq"
server_key: "ba690d5dd3012932b7d4119c92e355e2"
server_key_iv: "879673ba382ab9e5a64e2bba84167efb"
server_key_version: "encryptionkey@0"
I20260807 08:48:06.593116 15966 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786092453486433-4734-0/minicluster-data/ts-0/wal/instance:
uuid: "4f8ad558036e4bcbbace591712a92519"
format_stamp: "Formatted at 2026-08-07 08:48:06 on dist-test-slave-6zbq"
server_key: "ba690d5dd3012932b7d4119c92e355e2"
server_key_iv: "879673ba382ab9e5a64e2bba84167efb"
server_key_version: "encryptionkey@0"
I20260807 08:48:06.594226 15966 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 08:48:06.594884 15992 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:48:06.595036 15966 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 08:48:06.595114 15966 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786092453486433-4734-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786092453486433-4734-0/minicluster-data/ts-0/wal
uuid: "4f8ad558036e4bcbbace591712a92519"
format_stamp: "Formatted at 2026-08-07 08:48:06 on dist-test-slave-6zbq"
server_key: "ba690d5dd3012932b7d4119c92e355e2"
server_key_iv: "879673ba382ab9e5a64e2bba84167efb"
server_key_version: "encryptionkey@0"
I20260807 08:48:06.595165 15966 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786092453486433-4734-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786092453486433-4734-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786092453486433-4734-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:48:06.610709 15966 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:48:06.612383 15966 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:48:06.612483 15966 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:48:06.612882 15966 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:48:06.612920 15966 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:48:06.612941 15966 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:48:06.612953 15966 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:48:06.618997 15966 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.129:41925
I20260807 08:48:06.619158 16168 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.129:41925 every 8 connection(s)
I20260807 08:48:06.619359 15966 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786092453486433-4734-0/minicluster-data/ts-0/data/info.pb
Aug 07 08:48:06 dist-test-slave-6zbq krb5kdc[15775](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092486, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.159.129@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM
I20260807 08:48:06.621937 15948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:06.613310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.129:60651 (local address 127.4.159.190:45303)
0807 08:48:06.613380 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:06.613383 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:06.613637 (+ 254us) server_negotiation.cc:407] Connection header received
0807 08:48:06.614084 (+ 447us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:06.614086 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:06.614099 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:06.614175 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:06.614931 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:06.615753 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:06.617169 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:06.617472 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:06.617550 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:06.619929 (+ 2379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:48:06.619930 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:48:06.619931 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:48:06.619937 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:48:06.620955 (+ 1018us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:48:06.621293 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:48:06.621293 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:48:06.621294 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:48:06.621338 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:48:06.621487 (+ 149us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:48:06.621487 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:48:06.621488 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:48:06.621575 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:48:06.621617 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0807 08:48:06.621813 (+ 196us) server_negotiation.cc:299] Negotiation successful
0807 08:48:06.621874 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260807 08:48:06.622080 16004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:06.613277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:45303 (local address 127.4.159.129:60651)
0807 08:48:06.613535 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:06.613565 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:06.613988 (+ 423us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:06.614228 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:06.614232 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:06.614566 (+ 334us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:48:06.614831 (+ 265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:06.614837 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:06.615847 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:06.615848 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:06.617072 (+ 1224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:06.617076 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:06.617553 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:06.617554 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:06.617651 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:06.618141 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:48:06.618151 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:48:06.619860 (+ 1709us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:48:06.621021 (+ 1161us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:48:06.621027 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:48:06.621029 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:48:06.621241 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:48:06.621375 (+ 134us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:48:06.621375 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:48:06.621376 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:48:06.621455 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:48:06.621615 (+ 160us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:48:06.621617 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:48:06.621722 (+ 105us) client_negotiation.cc:769] Sending connection context
0807 08:48:06.621793 (+ 71us) client_negotiation.cc:240] Negotiation successful
0807 08:48:06.621902 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":72,"threads_started":1}
I20260807 08:48:06.622560 16170 heartbeater.cc:347] Connected to a master server at 127.4.159.190:45303
I20260807 08:48:06.622634 16170 heartbeater.cc:464] Registering TS with master...
I20260807 08:48:06.622797 16170 heartbeater.cc:511] Master 127.4.159.190:45303 requested a full tablet report, sending...
I20260807 08:48:06.623312 15852 ts_manager.cc:194] Registered new tserver with Master: 4f8ad558036e4bcbbace591712a92519 (127.4.159.129:41925)
I20260807 08:48:06.623919 15852 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.4.159.129@KRBTEST.COM'} at 127.4.159.129:60651
I20260807 08:48:06.625859 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 15966
I20260807 08:48:06.625946 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786092453486433-4734-0/minicluster-data/ts-0/wal/instance
I20260807 08:48:06.626122 4734 external_mini_cluster.cc:1428] Setting key 90432777f92b03189dfe3bb6b8c97fc8
I20260807 08:48:06.628682 15948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:06.623490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.129:46493 (local address 127.4.159.190:45303)
0807 08:48:06.623534 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:06.623536 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:06.623650 (+ 114us) server_negotiation.cc:407] Connection header received
0807 08:48:06.623699 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:06.623699 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:06.623710 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:06.623752 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:06.624032 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:06.624481 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:06.625131 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:06.625334 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:06.625387 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:06.626158 (+ 771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:48:06.626159 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:48:06.626159 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:48:06.626166 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:48:06.627555 (+ 1389us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:48:06.627875 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:48:06.627876 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:48:06.627876 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:48:06.627933 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:48:06.628081 (+ 148us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:48:06.628082 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:48:06.628082 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:48:06.628244 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:48:06.628311 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 08:48:06.628514 (+ 203us) server_negotiation.cc:299] Negotiation successful
0807 08:48:06.628594 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 08:48:06.628954 16178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:06.623443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:45303 (local address 127.4.159.129:46493)
0807 08:48:06.623588 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:06.623597 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:06.623646 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:06.623754 (+ 108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:06.623755 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:06.623902 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:48:06.623967 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:06.623969 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:06.624562 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:06.624563 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:06.625065 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:06.625066 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:06.625392 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:06.625393 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:06.625459 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:06.625783 (+ 324us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:48:06.625787 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:48:06.626086 (+ 299us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:48:06.627642 (+ 1556us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:48:06.627643 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:48:06.627643 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:48:06.627811 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:48:06.627987 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:48:06.627987 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:48:06.627988 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:48:06.628033 (+ 45us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:48:06.628341 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:48:06.628343 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:48:06.628401 (+ 58us) client_negotiation.cc:769] Sending connection context
0807 08:48:06.628754 (+ 353us) client_negotiation.cc:240] Negotiation successful
0807 08:48:06.628857 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":63,"threads_started":1}
WARNING: no policy specified for oryx/127.4.159.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.159.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.159.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786092453486433-4734-0/krb5kdc/oryx_127.4.159.130.keytab.
Entry for principal oryx/127.4.159.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786092453486433-4734-0/krb5kdc/oryx_127.4.159.130.keytab.
I20260807 08:48:06.648679 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786092453486433-4734-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786092453486433-4734-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786092453486433-4734-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786092453486433-4734-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.159.130:0
--local_ip_for_outbound_sockets=127.4.159.130
--webserver_interface=127.4.159.130
--webserver_port=0
--tserver_master_addrs=127.4.159.190:45303
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786092453486433-4734-0/krb5kdc/oryx_127.4.159.130.keytab
--principal=oryx/127.4.159.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:45103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:48:06.727543 16181 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:48:06.727744 16181 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:48:06.727775 16181 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:48:06.728916 16181 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:48:06.728955 16181 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:48:06.728981 16181 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 08:48:06.729015 16181 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.159.130
I20260807 08:48:06.730515 16181 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:45103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786092453486433-4734-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786092453486433-4734-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786092453486433-4734-0/krb5kdc/oryx_127.4.159.130.keytab
--principal=oryx/127.4.159.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.159.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786092453486433-4734-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.159.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.4.159.190:45303
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16181
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.159.130
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786092453486433-4734-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:48:06.730741 16181 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:48:06.730929 16181 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:48:06.735302 16189 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:48:06.735366 16187 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:48:06.735375 16181 server_base.cc:1034] running on GCE node
W20260807 08:48:06.735494 16196 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:48:06.735697 16181 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:48:06.735909 16181 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:48:06.737042 16181 hybrid_clock.cc:648] HybridClock initialized: now 1786092486737030 us; error 27 us; skew 500 ppm
Aug 07 08:48:06 dist-test-slave-6zbq krb5kdc[15775](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092486, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.159.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:48:06.739300 16181 init.cc:377] Logged in from keytab as oryx/127.4.159.130@KRBTEST.COM (short username oryx)
I20260807 08:48:06.739662 16181 webserver.cc:460] Webserver started at http://127.4.159.130:40933/ using document root <none> and password file <none>
I20260807 08:48:06.739852 16181 fs_manager.cc:359] Metadata directory not provided
I20260807 08:48:06.739897 16181 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:48:06.740013 16181 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:48:06.740918 16181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786092453486433-4734-0/minicluster-data/ts-1/data/instance:
uuid: "a6627f8faaf1423fb7504793451b18c3"
format_stamp: "Formatted at 2026-08-07 08:48:06 on dist-test-slave-6zbq"
server_key: "ab2977b800ddc38f188eb0bd0336a463"
server_key_iv: "b70810c2839e21ee18dccb64ad128607"
server_key_version: "encryptionkey@0"
I20260807 08:48:06.741250 16181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786092453486433-4734-0/minicluster-data/ts-1/wal/instance:
uuid: "a6627f8faaf1423fb7504793451b18c3"
format_stamp: "Formatted at 2026-08-07 08:48:06 on dist-test-slave-6zbq"
server_key: "ab2977b800ddc38f188eb0bd0336a463"
server_key_iv: "b70810c2839e21ee18dccb64ad128607"
server_key_version: "encryptionkey@0"
I20260807 08:48:06.742530 16181 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 08:48:06.743151 16211 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:48:06.743294 16181 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 08:48:06.743377 16181 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786092453486433-4734-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786092453486433-4734-0/minicluster-data/ts-1/wal
uuid: "a6627f8faaf1423fb7504793451b18c3"
format_stamp: "Formatted at 2026-08-07 08:48:06 on dist-test-slave-6zbq"
server_key: "ab2977b800ddc38f188eb0bd0336a463"
server_key_iv: "b70810c2839e21ee18dccb64ad128607"
server_key_version: "encryptionkey@0"
I20260807 08:48:06.743439 16181 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786092453486433-4734-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786092453486433-4734-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786092453486433-4734-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:48:06.753420 16181 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:48:06.755254 16181 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:48:06.755381 16181 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:48:06.755815 16181 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:48:06.755856 16181 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:48:06.755883 16181 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:48:06.755900 16181 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 08:48:06 dist-test-slave-6zbq krb5kdc[15775](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092486, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.159.130@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM
I20260807 08:48:06.762467 16181 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.130:44955
I20260807 08:48:06.762531 16383 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.130:44955 every 8 connection(s)
I20260807 08:48:06.762784 16181 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786092453486433-4734-0/minicluster-data/ts-1/data/info.pb
I20260807 08:48:06.764489 15948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:06.756266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.130:58035 (local address 127.4.159.190:45303)
0807 08:48:06.756317 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:06.756319 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:06.756524 (+ 205us) server_negotiation.cc:407] Connection header received
0807 08:48:06.756936 (+ 412us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:06.756937 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:06.756949 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:06.757016 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:06.757762 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:06.758531 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:06.759516 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:06.759806 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:06.759883 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:06.762150 (+ 2267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:48:06.762151 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:48:06.762152 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:48:06.762158 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:48:06.763512 (+ 1354us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:48:06.763847 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:48:06.763848 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:48:06.763848 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:48:06.763897 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:48:06.764059 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:48:06.764060 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:48:06.764060 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:48:06.764151 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:48:06.764193 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0807 08:48:06.764375 (+ 182us) server_negotiation.cc:299] Negotiation successful
0807 08:48:06.764429 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260807 08:48:06.764632 16224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:06.756267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:45303 (local address 127.4.159.130:58035)
0807 08:48:06.756408 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:06.756440 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:06.756852 (+ 412us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:06.757100 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:06.757103 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:06.757412 (+ 309us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:48:06.757666 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:06.757673 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:06.758620 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:06.758620 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:06.759448 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:06.759450 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:06.759905 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:06.759906 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:06.760032 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:06.760473 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:48:06.760482 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:48:06.762045 (+ 1563us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:48:06.763577 (+ 1532us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:48:06.763582 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:48:06.763583 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:48:06.763786 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:48:06.763943 (+ 157us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:48:06.763944 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:48:06.763944 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:48:06.764026 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:48:06.764194 (+ 168us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:48:06.764196 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:48:06.764291 (+ 95us) client_negotiation.cc:769] Sending connection context
0807 08:48:06.764361 (+ 70us) client_negotiation.cc:240] Negotiation successful
0807 08:48:06.764464 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":81,"threads_started":1}
I20260807 08:48:06.765065 16384 heartbeater.cc:347] Connected to a master server at 127.4.159.190:45303
I20260807 08:48:06.765154 16384 heartbeater.cc:464] Registering TS with master...
I20260807 08:48:06.765332 16384 heartbeater.cc:511] Master 127.4.159.190:45303 requested a full tablet report, sending...
I20260807 08:48:06.765687 15850 ts_manager.cc:194] Registered new tserver with Master: a6627f8faaf1423fb7504793451b18c3 (127.4.159.130:44955)
I20260807 08:48:06.766135 15850 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.4.159.130@KRBTEST.COM'} at 127.4.159.130:58035
I20260807 08:48:06.770454 15948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:06.766043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.130:33293 (local address 127.4.159.190:45303)
0807 08:48:06.766077 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:06.766079 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:06.766230 (+ 151us) server_negotiation.cc:407] Connection header received
0807 08:48:06.766251 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:06.766252 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:06.766260 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:06.766300 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:06.766670 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:06.767081 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:06.767688 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:06.767846 (+ 158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:06.767884 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:06.768540 (+ 656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:48:06.768541 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:48:06.768541 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:48:06.768547 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:48:06.769639 (+ 1092us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:48:06.769899 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:48:06.769899 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:48:06.769899 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:48:06.769939 (+ 40us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:48:06.770044 (+ 105us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:48:06.770045 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:48:06.770045 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:48:06.770154 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:48:06.770207 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0807 08:48:06.770306 (+ 99us) server_negotiation.cc:299] Negotiation successful
0807 08:48:06.770384 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260807 08:48:06.770478 16390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:06.766000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:45303 (local address 127.4.159.130:33293)
0807 08:48:06.766134 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:06.766142 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:06.766193 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:06.766315 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:06.766316 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:06.766515 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:48:06.766599 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:06.766601 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:06.767161 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:06.767162 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:06.767631 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:06.767633 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:06.767890 (+ 257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:06.767890 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:06.767933 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:06.768224 (+ 291us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:48:06.768227 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:48:06.768479 (+ 252us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:48:06.769702 (+ 1223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:48:06.769703 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:48:06.769703 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:48:06.769848 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:48:06.769973 (+ 125us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:48:06.769974 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:48:06.769974 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:48:06.770014 (+ 40us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:48:06.770201 (+ 187us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:48:06.770201 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:48:06.770233 (+ 32us) client_negotiation.cc:769] Sending connection context
0807 08:48:06.770290 (+ 57us) client_negotiation.cc:240] Negotiation successful
0807 08:48:06.770390 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":66,"threads_started":1}
I20260807 08:48:06.772734 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 16181
I20260807 08:48:06.772842 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786092453486433-4734-0/minicluster-data/ts-1/wal/instance
I20260807 08:48:06.773005 4734 external_mini_cluster.cc:1428] Setting key 81035d922af7e9a532a49a97291c8e49
WARNING: no policy specified for oryx/127.4.159.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.159.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.159.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786092453486433-4734-0/krb5kdc/oryx_127.4.159.131.keytab.
Entry for principal oryx/127.4.159.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786092453486433-4734-0/krb5kdc/oryx_127.4.159.131.keytab.
I20260807 08:48:06.794678 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786092453486433-4734-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786092453486433-4734-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786092453486433-4734-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786092453486433-4734-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.159.131:0
--local_ip_for_outbound_sockets=127.4.159.131
--webserver_interface=127.4.159.131
--webserver_port=0
--tserver_master_addrs=127.4.159.190:45303
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786092453486433-4734-0/krb5kdc/oryx_127.4.159.131.keytab
--principal=oryx/127.4.159.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:45103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:48:06.872179 16393 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:48:06.872354 16393 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:48:06.872373 16393 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:48:06.873423 16393 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:48:06.873454 16393 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:48:06.873472 16393 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 08:48:06.873488 16393 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.159.131
I20260807 08:48:06.874922 16393 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:45103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786092453486433-4734-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786092453486433-4734-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786092453486433-4734-0/krb5kdc/oryx_127.4.159.131.keytab
--principal=oryx/127.4.159.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.159.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786092453486433-4734-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.159.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.4.159.190:45303
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16393
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.159.131
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786092453486433-4734-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:48:06.875090 16393 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:48:06.875250 16393 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:48:06.879635 16404 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:48:06.879606 16403 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:48:06.879665 16393 server_base.cc:1034] running on GCE node
W20260807 08:48:06.879592 16407 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:48:06.879993 16393 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:48:06.880200 16393 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:48:06.881335 16393 hybrid_clock.cc:648] HybridClock initialized: now 1786092486881320 us; error 32 us; skew 500 ppm
Aug 07 08:48:06 dist-test-slave-6zbq krb5kdc[15775](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092486, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.159.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:48:06.883628 16393 init.cc:377] Logged in from keytab as oryx/127.4.159.131@KRBTEST.COM (short username oryx)
I20260807 08:48:06.884009 16393 webserver.cc:460] Webserver started at http://127.4.159.131:33081/ using document root <none> and password file <none>
I20260807 08:48:06.884187 16393 fs_manager.cc:359] Metadata directory not provided
I20260807 08:48:06.884225 16393 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:48:06.884311 16393 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:48:06.885187 16393 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786092453486433-4734-0/minicluster-data/ts-2/data/instance:
uuid: "9bcc8a5781ff4eb5bcbe77690fd0c0d4"
format_stamp: "Formatted at 2026-08-07 08:48:06 on dist-test-slave-6zbq"
server_key: "263481e8d58767fbb1236f51bfb99c4c"
server_key_iv: "298f398ece7ffaea5dc5de59781fcf04"
server_key_version: "encryptionkey@0"
I20260807 08:48:06.885462 16393 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786092453486433-4734-0/minicluster-data/ts-2/wal/instance:
uuid: "9bcc8a5781ff4eb5bcbe77690fd0c0d4"
format_stamp: "Formatted at 2026-08-07 08:48:06 on dist-test-slave-6zbq"
server_key: "263481e8d58767fbb1236f51bfb99c4c"
server_key_iv: "298f398ece7ffaea5dc5de59781fcf04"
server_key_version: "encryptionkey@0"
I20260807 08:48:06.886610 16393 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260807 08:48:06.887283 16422 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:48:06.887458 16393 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 08:48:06.887531 16393 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786092453486433-4734-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786092453486433-4734-0/minicluster-data/ts-2/wal
uuid: "9bcc8a5781ff4eb5bcbe77690fd0c0d4"
format_stamp: "Formatted at 2026-08-07 08:48:06 on dist-test-slave-6zbq"
server_key: "263481e8d58767fbb1236f51bfb99c4c"
server_key_iv: "298f398ece7ffaea5dc5de59781fcf04"
server_key_version: "encryptionkey@0"
I20260807 08:48:06.887581 16393 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786092453486433-4734-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786092453486433-4734-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786092453486433-4734-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:48:06.895610 16393 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:48:06.897095 16393 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:48:06.897192 16393 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:48:06.897578 16393 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:48:06.897619 16393 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:48:06.897636 16393 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:48:06.897647 16393 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:48:06.903475 16393 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.131:42305
I20260807 08:48:06.903548 16606 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.131:42305 every 8 connection(s)
Aug 07 08:48:06 dist-test-slave-6zbq krb5kdc[15775](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092486, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.159.131@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM
I20260807 08:48:06.903817 16393 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786092453486433-4734-0/minicluster-data/ts-2/data/info.pb
I20260807 08:48:06.906683 15948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:06.897980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.131:42109 (local address 127.4.159.190:45303)
0807 08:48:06.898044 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:06.898047 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:06.898188 (+ 141us) server_negotiation.cc:407] Connection header received
0807 08:48:06.898642 (+ 454us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:06.898644 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:06.898657 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:06.898731 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:06.899421 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:06.900143 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:06.901561 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:06.901851 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:06.901923 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:06.904193 (+ 2270us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:48:06.904195 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:48:06.904195 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:48:06.904203 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:48:06.905409 (+ 1206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:48:06.905764 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:48:06.905765 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:48:06.905766 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:48:06.905828 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:48:06.906039 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:48:06.906040 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:48:06.906041 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:48:06.906150 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:48:06.906212 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 08:48:06.906511 (+ 299us) server_negotiation.cc:299] Negotiation successful
0807 08:48:06.906590 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260807 08:48:06.906809 16436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:06.897937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:45303 (local address 127.4.159.131:42109)
0807 08:48:06.898077 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:06.898113 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:06.898551 (+ 438us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:06.898793 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:06.898797 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:06.899073 (+ 276us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:48:06.899327 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:06.899332 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:06.900245 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:06.900246 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:06.901470 (+ 1224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:06.901474 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:06.901927 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:06.901928 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:06.902060 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:06.902498 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:48:06.902507 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:48:06.904107 (+ 1600us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:48:06.905482 (+ 1375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:48:06.905488 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:48:06.905490 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:48:06.905696 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:48:06.905883 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:48:06.905884 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:48:06.905884 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:48:06.905986 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:48:06.906204 (+ 218us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:48:06.906210 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:48:06.906334 (+ 124us) client_negotiation.cc:769] Sending connection context
0807 08:48:06.906466 (+ 132us) client_negotiation.cc:240] Negotiation successful
0807 08:48:06.906606 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":78,"threads_started":1}
I20260807 08:48:06.907255 16610 heartbeater.cc:347] Connected to a master server at 127.4.159.190:45303
I20260807 08:48:06.907334 16610 heartbeater.cc:464] Registering TS with master...
I20260807 08:48:06.907500 16610 heartbeater.cc:511] Master 127.4.159.190:45303 requested a full tablet report, sending...
I20260807 08:48:06.907887 15852 ts_manager.cc:194] Registered new tserver with Master: 9bcc8a5781ff4eb5bcbe77690fd0c0d4 (127.4.159.131:42305)
I20260807 08:48:06.908354 15852 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.4.159.131@KRBTEST.COM'} at 127.4.159.131:42109
I20260807 08:48:06.908622 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 16393
I20260807 08:48:06.908747 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786092453486433-4734-0/minicluster-data/ts-2/wal/instance
I20260807 08:48:06.908943 4734 external_mini_cluster.cc:1428] Setting key 0c1eabc2ffad4dd19b09457b9593b666
I20260807 08:48:06.909804 4734 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 08:48:06.913491 15948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:06.908208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.131:52101 (local address 127.4.159.190:45303)
0807 08:48:06.908247 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:06.908249 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:06.908308 (+ 59us) server_negotiation.cc:407] Connection header received
0807 08:48:06.908354 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:06.908354 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:06.908364 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:06.908418 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:06.908776 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:06.909304 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:06.910252 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:06.910476 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:06.910527 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:06.911375 (+ 848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:48:06.911376 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:48:06.911376 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:48:06.911382 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:48:06.912516 (+ 1134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:48:06.912816 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:48:06.912817 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:48:06.912817 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:48:06.912869 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:48:06.913048 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:48:06.913049 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:48:06.913049 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:48:06.913177 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:48:06.913233 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 08:48:06.913380 (+ 147us) server_negotiation.cc:299] Negotiation successful
0807 08:48:06.913434 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 08:48:06.913548 16617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:06.908140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:45303 (local address 127.4.159.131:52101)
0807 08:48:06.908257 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:06.908266 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:06.908318 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:06.908444 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:06.908445 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:06.908617 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:48:06.908672 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:06.908674 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:06.909379 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:06.909380 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:06.910051 (+ 671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:06.910054 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:06.910541 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:06.910542 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:06.910617 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:06.910954 (+ 337us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:48:06.910958 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:48:06.911294 (+ 336us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:48:06.912588 (+ 1294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:48:06.912589 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:48:06.912590 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:48:06.912755 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:48:06.912920 (+ 165us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:48:06.912921 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:48:06.912921 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:48:06.912977 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:48:06.913241 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:48:06.913242 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:48:06.913288 (+ 46us) client_negotiation.cc:769] Sending connection context
0807 08:48:06.913361 (+ 73us) client_negotiation.cc:240] Negotiation successful
0807 08:48:06.913463 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":63,"threads_started":1}
Aug 07 08:48:06 dist-test-slave-6zbq krb5kdc[15775](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092486, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 08:48:06 dist-test-slave-6zbq krb5kdc[15775](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.4.159.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:48:06 dist-test-slave-6zbq krb5kdc[15775](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.4.159.190@KRBTEST.COM, Server not found in Kerberos database
W20260807 08:48:06.931106 15948 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:48:06.926555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52322 (local address 127.4.159.190:45303)
0807 08:48:06.926594 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:06.926595 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:06.926628 (+ 33us) server_negotiation.cc:407] Connection header received
0807 08:48:06.926680 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:06.926681 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:06.926687 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:06.926728 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:06.927246 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:06.927686 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:06.928340 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:06.928523 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:06.928570 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:06.931051 (+ 2481us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52322: BlockingRecv error: recv got EOF from 127.0.0.1:52322 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
Aug 07 08:48:06 dist-test-slave-6zbq krb5kdc[15775](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092486, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM
I20260807 08:48:06.937019 15948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:06.931767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52334 (local address 127.4.159.190:45303)
0807 08:48:06.931805 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:06.931806 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:06.931849 (+ 43us) server_negotiation.cc:407] Connection header received
0807 08:48:06.931898 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:06.931898 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:06.931905 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:06.931944 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:06.932319 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:06.932733 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:06.933300 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:06.933456 (+ 156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:06.933493 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:06.935192 (+ 1699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:48:06.935193 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:48:06.935193 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:48:06.935199 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:48:06.936319 (+ 1120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:48:06.936556 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:48:06.936556 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:48:06.936557 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:48:06.936598 (+ 41us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:48:06.936710 (+ 112us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:48:06.936711 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:48:06.936711 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:48:06.936789 (+ 78us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:48:06.936850 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 08:48:06.936919 (+ 69us) server_negotiation.cc:299] Negotiation successful
0807 08:48:06.936968 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260807 08:48:06.938205 15852 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52334:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 08:48:06.939694 15852 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:48:06.945493 16640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:06.941541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.130:44955 (local address 127.0.0.1:60196)
0807 08:48:06.941739 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:06.941789 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:06.941850 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:06.942206 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:06.942207 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:06.942209 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:48:06.942320 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:06.942322 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:06.943229 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:06.943230 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:06.944355 (+ 1125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:06.944358 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:06.945210 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:06.945218 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:06.945296 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:06.945309 (+ 13us) client_negotiation.cc:769] Sending connection context
0807 08:48:06.945379 (+ 70us) client_negotiation.cc:240] Negotiation successful
0807 08:48:06.945426 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":51,"threads_started":1}
I20260807 08:48:06.945539 16645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:06.941747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60196 (local address 127.4.159.130:44955)
0807 08:48:06.942000 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:06.942004 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:06.942013 (+ 9us) server_negotiation.cc:407] Connection header received
0807 08:48:06.942052 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:06.942054 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:06.942069 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:06.942138 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:48:06.942414 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:06.943146 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:06.944471 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:06.945127 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:06.945198 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:06.945272 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 08:48:06.945399 (+ 127us) server_negotiation.cc:299] Negotiation successful
0807 08:48:06.945438 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":225,"thread_start_us":63,"threads_started":1}
I20260807 08:48:06.945943 16638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:06.941541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.129:41925 (local address 127.0.0.1:57072)
0807 08:48:06.941883 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:06.941894 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:06.941945 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:06.942134 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:06.942137 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:06.942140 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:48:06.942202 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:06.942204 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:06.943590 (+ 1386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:06.943591 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:06.944608 (+ 1017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:06.944610 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:06.945502 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:06.945504 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:06.945828 (+ 324us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:06.945833 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 08:48:06.945867 (+ 34us) client_negotiation.cc:240] Negotiation successful
0807 08:48:06.945897 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":329,"mutex_wait_us":4,"thread_start_us":59,"threads_started":1}
I20260807 08:48:06.946272 16643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:06.941724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57072 (local address 127.4.159.129:41925)
0807 08:48:06.941918 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:06.941922 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:06.941934 (+ 12us) server_negotiation.cc:407] Connection header received
0807 08:48:06.942021 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:06.942023 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:06.942036 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:06.942272 (+ 236us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:48:06.942394 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:06.943504 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:06.944698 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:06.945407 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:06.946045 (+ 638us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:06.946139 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 08:48:06.946165 (+ 26us) server_negotiation.cc:299] Negotiation successful
0807 08:48:06.946203 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":75,"threads_started":1}
I20260807 08:48:06.946204 16286 tablet_service.cc:1467] Processing CreateTablet for tablet bff437cdbea54a67b64ae40a7ca8d681 (DEFAULT_TABLE table=test-table [id=557cde2f3f854de9a5d34b4ce525f7ef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:48:06.946623 16636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:06.941554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.131:42305 (local address 127.0.0.1:41800)
0807 08:48:06.942031 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:06.942038 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:06.942085 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:06.942231 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:06.942232 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:06.942233 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:48:06.942265 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:06.942266 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:06.944692 (+ 2426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:06.944693 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:06.945431 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:06.945434 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:06.946406 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:06.946407 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:06.946466 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:06.946470 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 08:48:06.946524 (+ 54us) client_negotiation.cc:240] Negotiation successful
0807 08:48:06.946560 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":92,"threads_started":1}
I20260807 08:48:06.946631 16286 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bff437cdbea54a67b64ae40a7ca8d681. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:48:06.946667 16646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:06.941745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41800 (local address 127.4.159.131:42305)
0807 08:48:06.941901 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:06.941906 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:06.942099 (+ 193us) server_negotiation.cc:407] Connection header received
0807 08:48:06.942137 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:06.942139 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:06.942152 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:06.942224 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:48:06.942401 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:06.943510 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:06.945517 (+ 2007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:06.946210 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:06.946277 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:06.946370 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 08:48:06.946538 (+ 168us) server_negotiation.cc:299] Negotiation successful
0807 08:48:06.946575 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":75,"threads_started":1}
I20260807 08:48:06.947250 16504 tablet_service.cc:1467] Processing CreateTablet for tablet bff437cdbea54a67b64ae40a7ca8d681 (DEFAULT_TABLE table=test-table [id=557cde2f3f854de9a5d34b4ce525f7ef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:48:06.947525 16504 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bff437cdbea54a67b64ae40a7ca8d681. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:48:06.949631 16648 tablet_bootstrap.cc:492] T bff437cdbea54a67b64ae40a7ca8d681 P a6627f8faaf1423fb7504793451b18c3: Bootstrap starting.
I20260807 08:48:06.950192 16649 tablet_bootstrap.cc:492] T bff437cdbea54a67b64ae40a7ca8d681 P 9bcc8a5781ff4eb5bcbe77690fd0c0d4: Bootstrap starting.
I20260807 08:48:06.950258 16648 tablet_bootstrap.cc:654] T bff437cdbea54a67b64ae40a7ca8d681 P a6627f8faaf1423fb7504793451b18c3: Neither blocks nor log segments found. Creating new log.
I20260807 08:48:06.950413 16068 tablet_service.cc:1467] Processing CreateTablet for tablet bff437cdbea54a67b64ae40a7ca8d681 (DEFAULT_TABLE table=test-table [id=557cde2f3f854de9a5d34b4ce525f7ef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:48:06.950632 16648 log.cc:824] T bff437cdbea54a67b64ae40a7ca8d681 P a6627f8faaf1423fb7504793451b18c3: Log is configured to *not* fsync() on all Append() calls
I20260807 08:48:06.950680 16068 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bff437cdbea54a67b64ae40a7ca8d681. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:48:06.950831 16649 tablet_bootstrap.cc:654] T bff437cdbea54a67b64ae40a7ca8d681 P 9bcc8a5781ff4eb5bcbe77690fd0c0d4: Neither blocks nor log segments found. Creating new log.
I20260807 08:48:06.951063 16649 log.cc:824] T bff437cdbea54a67b64ae40a7ca8d681 P 9bcc8a5781ff4eb5bcbe77690fd0c0d4: Log is configured to *not* fsync() on all Append() calls
I20260807 08:48:06.951383 16648 tablet_bootstrap.cc:492] T bff437cdbea54a67b64ae40a7ca8d681 P a6627f8faaf1423fb7504793451b18c3: No bootstrap required, opened a new log
I20260807 08:48:06.951426 16648 ts_tablet_manager.cc:1397] T bff437cdbea54a67b64ae40a7ca8d681 P a6627f8faaf1423fb7504793451b18c3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 08:48:06.951784 16649 tablet_bootstrap.cc:492] T bff437cdbea54a67b64ae40a7ca8d681 P 9bcc8a5781ff4eb5bcbe77690fd0c0d4: No bootstrap required, opened a new log
I20260807 08:48:06.951838 16649 ts_tablet_manager.cc:1397] T bff437cdbea54a67b64ae40a7ca8d681 P 9bcc8a5781ff4eb5bcbe77690fd0c0d4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 08:48:06.953254 16648 raft_consensus.cc:348] T bff437cdbea54a67b64ae40a7ca8d681 P a6627f8faaf1423fb7504793451b18c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f8ad558036e4bcbbace591712a92519" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 41925 } } peers { permanent_uuid: "a6627f8faaf1423fb7504793451b18c3" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 44955 } } peers { permanent_uuid: "9bcc8a5781ff4eb5bcbe77690fd0c0d4" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 42305 } }
I20260807 08:48:06.953404 16648 raft_consensus.cc:374] T bff437cdbea54a67b64ae40a7ca8d681 P a6627f8faaf1423fb7504793451b18c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:48:06.953440 16648 raft_consensus.cc:729] T bff437cdbea54a67b64ae40a7ca8d681 P a6627f8faaf1423fb7504793451b18c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6627f8faaf1423fb7504793451b18c3, State: Initialized, Role: FOLLOWER
I20260807 08:48:06.953596 16648 consensus_queue.cc:260] T bff437cdbea54a67b64ae40a7ca8d681 P a6627f8faaf1423fb7504793451b18c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f8ad558036e4bcbbace591712a92519" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 41925 } } peers { permanent_uuid: "a6627f8faaf1423fb7504793451b18c3" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 44955 } } peers { permanent_uuid: "9bcc8a5781ff4eb5bcbe77690fd0c0d4" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 42305 } }
I20260807 08:48:06.953796 16649 raft_consensus.cc:348] T bff437cdbea54a67b64ae40a7ca8d681 P 9bcc8a5781ff4eb5bcbe77690fd0c0d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f8ad558036e4bcbbace591712a92519" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 41925 } } peers { permanent_uuid: "a6627f8faaf1423fb7504793451b18c3" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 44955 } } peers { permanent_uuid: "9bcc8a5781ff4eb5bcbe77690fd0c0d4" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 42305 } }
I20260807 08:48:06.953907 16648 ts_tablet_manager.cc:1428] T bff437cdbea54a67b64ae40a7ca8d681 P a6627f8faaf1423fb7504793451b18c3: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260807 08:48:06.953946 16649 raft_consensus.cc:374] T bff437cdbea54a67b64ae40a7ca8d681 P 9bcc8a5781ff4eb5bcbe77690fd0c0d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:48:06.953951 16653 tablet_bootstrap.cc:492] T bff437cdbea54a67b64ae40a7ca8d681 P 4f8ad558036e4bcbbace591712a92519: Bootstrap starting.
I20260807 08:48:06.953987 16649 raft_consensus.cc:729] T bff437cdbea54a67b64ae40a7ca8d681 P 9bcc8a5781ff4eb5bcbe77690fd0c0d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9bcc8a5781ff4eb5bcbe77690fd0c0d4, State: Initialized, Role: FOLLOWER
I20260807 08:48:06.954015 16384 heartbeater.cc:503] Master 127.4.159.190:45303 was elected leader, sending a full tablet report...
I20260807 08:48:06.954097 16649 consensus_queue.cc:260] T bff437cdbea54a67b64ae40a7ca8d681 P 9bcc8a5781ff4eb5bcbe77690fd0c0d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f8ad558036e4bcbbace591712a92519" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 41925 } } peers { permanent_uuid: "a6627f8faaf1423fb7504793451b18c3" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 44955 } } peers { permanent_uuid: "9bcc8a5781ff4eb5bcbe77690fd0c0d4" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 42305 } }
I20260807 08:48:06.954376 16649 ts_tablet_manager.cc:1428] T bff437cdbea54a67b64ae40a7ca8d681 P 9bcc8a5781ff4eb5bcbe77690fd0c0d4: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 08:48:06.954485 16610 heartbeater.cc:503] Master 127.4.159.190:45303 was elected leader, sending a full tablet report...
I20260807 08:48:06.954597 16653 tablet_bootstrap.cc:654] T bff437cdbea54a67b64ae40a7ca8d681 P 4f8ad558036e4bcbbace591712a92519: Neither blocks nor log segments found. Creating new log.
I20260807 08:48:06.954895 16653 log.cc:824] T bff437cdbea54a67b64ae40a7ca8d681 P 4f8ad558036e4bcbbace591712a92519: Log is configured to *not* fsync() on all Append() calls
I20260807 08:48:06.955634 16653 tablet_bootstrap.cc:492] T bff437cdbea54a67b64ae40a7ca8d681 P 4f8ad558036e4bcbbace591712a92519: No bootstrap required, opened a new log
I20260807 08:48:06.955693 16653 ts_tablet_manager.cc:1397] T bff437cdbea54a67b64ae40a7ca8d681 P 4f8ad558036e4bcbbace591712a92519: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 08:48:06.956997 16653 raft_consensus.cc:348] T bff437cdbea54a67b64ae40a7ca8d681 P 4f8ad558036e4bcbbace591712a92519 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f8ad558036e4bcbbace591712a92519" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 41925 } } peers { permanent_uuid: "a6627f8faaf1423fb7504793451b18c3" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 44955 } } peers { permanent_uuid: "9bcc8a5781ff4eb5bcbe77690fd0c0d4" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 42305 } }
I20260807 08:48:06.957111 16653 raft_consensus.cc:374] T bff437cdbea54a67b64ae40a7ca8d681 P 4f8ad558036e4bcbbace591712a92519 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:48:06.957130 16653 raft_consensus.cc:729] T bff437cdbea54a67b64ae40a7ca8d681 P 4f8ad558036e4bcbbace591712a92519 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f8ad558036e4bcbbace591712a92519, State: Initialized, Role: FOLLOWER
I20260807 08:48:06.957206 16653 consensus_queue.cc:260] T bff437cdbea54a67b64ae40a7ca8d681 P 4f8ad558036e4bcbbace591712a92519 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f8ad558036e4bcbbace591712a92519" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 41925 } } peers { permanent_uuid: "a6627f8faaf1423fb7504793451b18c3" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 44955 } } peers { permanent_uuid: "9bcc8a5781ff4eb5bcbe77690fd0c0d4" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 42305 } }
I20260807 08:48:06.957398 16653 ts_tablet_manager.cc:1428] T bff437cdbea54a67b64ae40a7ca8d681 P 4f8ad558036e4bcbbace591712a92519: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 08:48:06.957463 16170 heartbeater.cc:503] Master 127.4.159.190:45303 was elected leader, sending a full tablet report...
W20260807 08:48:07.013841 16385 tablet.cc:2369] T bff437cdbea54a67b64ae40a7ca8d681 P a6627f8faaf1423fb7504793451b18c3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 08:48:07.039546 16658 raft_consensus.cc:482] T bff437cdbea54a67b64ae40a7ca8d681 P 9bcc8a5781ff4eb5bcbe77690fd0c0d4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:48:07.039662 16658 raft_consensus.cc:504] T bff437cdbea54a67b64ae40a7ca8d681 P 9bcc8a5781ff4eb5bcbe77690fd0c0d4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f8ad558036e4bcbbace591712a92519" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 41925 } } peers { permanent_uuid: "a6627f8faaf1423fb7504793451b18c3" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 44955 } } peers { permanent_uuid: "9bcc8a5781ff4eb5bcbe77690fd0c0d4" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 42305 } }
I20260807 08:48:07.039963 16658 leader_election.cc:288] T bff437cdbea54a67b64ae40a7ca8d681 P 9bcc8a5781ff4eb5bcbe77690fd0c0d4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f8ad558036e4bcbbace591712a92519 (127.4.159.129:41925), a6627f8faaf1423fb7504793451b18c3 (127.4.159.130:44955)
I20260807 08:48:07.042658 16643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:07.040302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.131:41943 (local address 127.4.159.129:41925)
0807 08:48:07.040347 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:07.040349 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:07.040356 (+ 7us) server_negotiation.cc:407] Connection header received
0807 08:48:07.040382 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:07.040383 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:07.040391 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:07.040446 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:48:07.040585 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:07.041066 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:07.041969 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:07.042321 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:07.042402 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:07.042412 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0807 08:48:07.042555 (+ 143us) server_negotiation.cc:299] Negotiation successful
0807 08:48:07.042591 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260807 08:48:07.042670 16436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:07.040162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.129:41925 (local address 127.4.159.131:41943)
0807 08:48:07.040225 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:07.040239 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:07.040306 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:07.040456 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:07.040457 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:07.040458 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:48:07.040527 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:07.040529 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:07.041153 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:07.041154 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:07.041880 (+ 726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:07.041882 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:07.042407 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:07.042408 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:07.042494 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:07.042498 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 08:48:07.042543 (+ 45us) client_negotiation.cc:240] Negotiation successful
0807 08:48:07.042582 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260807 08:48:07.042825 16645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:07.040281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.131:36031 (local address 127.4.159.130:44955)
0807 08:48:07.040328 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:07.040330 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:07.040415 (+ 85us) server_negotiation.cc:407] Connection header received
0807 08:48:07.040468 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:07.040469 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:07.040479 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:07.040524 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:48:07.040671 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:07.041227 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:07.042143 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:07.042528 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:07.042589 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:07.042596 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0807 08:48:07.042726 (+ 130us) server_negotiation.cc:299] Negotiation successful
0807 08:48:07.042759 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 08:48:07.042855 16665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:07.040162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.130:44955 (local address 127.4.159.131:36031)
0807 08:48:07.040347 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:07.040356 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:07.040415 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:07.040534 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:07.040535 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:07.040536 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:48:07.040615 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:07.040616 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:07.041340 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:07.041341 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:07.042053 (+ 712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:07.042055 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:07.042596 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:07.042597 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:07.042669 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:07.042671 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 08:48:07.042715 (+ 44us) client_negotiation.cc:240] Negotiation successful
0807 08:48:07.042752 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"mutex_wait_us":21,"thread_start_us":94,"threads_started":1}
I20260807 08:48:07.042994 16097 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bff437cdbea54a67b64ae40a7ca8d681" candidate_uuid: "9bcc8a5781ff4eb5bcbe77690fd0c0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f8ad558036e4bcbbace591712a92519" is_pre_election: true
I20260807 08:48:07.043145 16097 raft_consensus.cc:2454] T bff437cdbea54a67b64ae40a7ca8d681 P 4f8ad558036e4bcbbace591712a92519 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9bcc8a5781ff4eb5bcbe77690fd0c0d4 in term 0.
I20260807 08:48:07.043123 16317 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bff437cdbea54a67b64ae40a7ca8d681" candidate_uuid: "9bcc8a5781ff4eb5bcbe77690fd0c0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6627f8faaf1423fb7504793451b18c3" is_pre_election: true
I20260807 08:48:07.043270 16317 raft_consensus.cc:2454] T bff437cdbea54a67b64ae40a7ca8d681 P a6627f8faaf1423fb7504793451b18c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9bcc8a5781ff4eb5bcbe77690fd0c0d4 in term 0.
I20260807 08:48:07.043346 16426 leader_election.cc:302] T bff437cdbea54a67b64ae40a7ca8d681 P 9bcc8a5781ff4eb5bcbe77690fd0c0d4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f8ad558036e4bcbbace591712a92519, 9bcc8a5781ff4eb5bcbe77690fd0c0d4; no voters:
I20260807 08:48:07.043473 16658 raft_consensus.cc:2781] T bff437cdbea54a67b64ae40a7ca8d681 P 9bcc8a5781ff4eb5bcbe77690fd0c0d4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:48:07.043537 16658 raft_consensus.cc:482] T bff437cdbea54a67b64ae40a7ca8d681 P 9bcc8a5781ff4eb5bcbe77690fd0c0d4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:48:07.043563 16658 raft_consensus.cc:3037] T bff437cdbea54a67b64ae40a7ca8d681 P 9bcc8a5781ff4eb5bcbe77690fd0c0d4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:48:07.044456 16658 raft_consensus.cc:504] T bff437cdbea54a67b64ae40a7ca8d681 P 9bcc8a5781ff4eb5bcbe77690fd0c0d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f8ad558036e4bcbbace591712a92519" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 41925 } } peers { permanent_uuid: "a6627f8faaf1423fb7504793451b18c3" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 44955 } } peers { permanent_uuid: "9bcc8a5781ff4eb5bcbe77690fd0c0d4" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 42305 } }
I20260807 08:48:07.044582 16658 leader_election.cc:288] T bff437cdbea54a67b64ae40a7ca8d681 P 9bcc8a5781ff4eb5bcbe77690fd0c0d4 [CANDIDATE]: Term 1 election: Requested vote from peers 4f8ad558036e4bcbbace591712a92519 (127.4.159.129:41925), a6627f8faaf1423fb7504793451b18c3 (127.4.159.130:44955)
I20260807 08:48:07.044727 16317 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bff437cdbea54a67b64ae40a7ca8d681" candidate_uuid: "9bcc8a5781ff4eb5bcbe77690fd0c0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6627f8faaf1423fb7504793451b18c3"
I20260807 08:48:07.044729 16097 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bff437cdbea54a67b64ae40a7ca8d681" candidate_uuid: "9bcc8a5781ff4eb5bcbe77690fd0c0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f8ad558036e4bcbbace591712a92519"
I20260807 08:48:07.044806 16317 raft_consensus.cc:3037] T bff437cdbea54a67b64ae40a7ca8d681 P a6627f8faaf1423fb7504793451b18c3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:48:07.044807 16097 raft_consensus.cc:3037] T bff437cdbea54a67b64ae40a7ca8d681 P 4f8ad558036e4bcbbace591712a92519 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:48:07.045462 16097 raft_consensus.cc:2454] T bff437cdbea54a67b64ae40a7ca8d681 P 4f8ad558036e4bcbbace591712a92519 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9bcc8a5781ff4eb5bcbe77690fd0c0d4 in term 1.
I20260807 08:48:07.045486 16317 raft_consensus.cc:2454] T bff437cdbea54a67b64ae40a7ca8d681 P a6627f8faaf1423fb7504793451b18c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9bcc8a5781ff4eb5bcbe77690fd0c0d4 in term 1.
I20260807 08:48:07.045639 16425 leader_election.cc:302] T bff437cdbea54a67b64ae40a7ca8d681 P 9bcc8a5781ff4eb5bcbe77690fd0c0d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9bcc8a5781ff4eb5bcbe77690fd0c0d4, a6627f8faaf1423fb7504793451b18c3; no voters:
I20260807 08:48:07.045758 16658 raft_consensus.cc:2781] T bff437cdbea54a67b64ae40a7ca8d681 P 9bcc8a5781ff4eb5bcbe77690fd0c0d4 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:48:07.045867 16658 raft_consensus.cc:686] T bff437cdbea54a67b64ae40a7ca8d681 P 9bcc8a5781ff4eb5bcbe77690fd0c0d4 [term 1 LEADER]: Becoming Leader. State: Replica: 9bcc8a5781ff4eb5bcbe77690fd0c0d4, State: Running, Role: LEADER
I20260807 08:48:07.045945 16658 consensus_queue.cc:237] T bff437cdbea54a67b64ae40a7ca8d681 P 9bcc8a5781ff4eb5bcbe77690fd0c0d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f8ad558036e4bcbbace591712a92519" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 41925 } } peers { permanent_uuid: "a6627f8faaf1423fb7504793451b18c3" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 44955 } } peers { permanent_uuid: "9bcc8a5781ff4eb5bcbe77690fd0c0d4" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 42305 } }
I20260807 08:48:07.046768 15852 catalog_manager.cc:5495] T bff437cdbea54a67b64ae40a7ca8d681 P 9bcc8a5781ff4eb5bcbe77690fd0c0d4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9bcc8a5781ff4eb5bcbe77690fd0c0d4 (127.4.159.131). New cstate: current_term: 1 leader_uuid: "9bcc8a5781ff4eb5bcbe77690fd0c0d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f8ad558036e4bcbbace591712a92519" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 41925 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a6627f8faaf1423fb7504793451b18c3" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 44955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9bcc8a5781ff4eb5bcbe77690fd0c0d4" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 42305 } health_report { overall_health: HEALTHY } } }
I20260807 08:48:07.059231 16646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:07.057290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41808 (local address 127.4.159.131:42305)
0807 08:48:07.057341 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:07.057343 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:07.057350 (+ 7us) server_negotiation.cc:407] Connection header received
0807 08:48:07.057396 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:07.057396 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:07.057411 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:07.057461 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:48:07.057560 (+ 99us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:07.057990 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:07.058675 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:07.058846 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:07.058891 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:07.058996 (+ 105us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:48:07.059081 (+ 85us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:48:07.059132 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0807 08:48:07.059142 (+ 10us) server_negotiation.cc:299] Negotiation successful
0807 08:48:07.059157 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260807 08:48:07.060842 16317 raft_consensus.cc:1261] T bff437cdbea54a67b64ae40a7ca8d681 P a6627f8faaf1423fb7504793451b18c3 [term 1 FOLLOWER]: Refusing update from remote peer 9bcc8a5781ff4eb5bcbe77690fd0c0d4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:48:07.060842 16097 raft_consensus.cc:1261] T bff437cdbea54a67b64ae40a7ca8d681 P 4f8ad558036e4bcbbace591712a92519 [term 1 FOLLOWER]: Refusing update from remote peer 9bcc8a5781ff4eb5bcbe77690fd0c0d4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:48:07.061161 16666 consensus_queue.cc:1035] T bff437cdbea54a67b64ae40a7ca8d681 P 9bcc8a5781ff4eb5bcbe77690fd0c0d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f8ad558036e4bcbbace591712a92519" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 41925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:48:07.061302 16658 consensus_queue.cc:1035] T bff437cdbea54a67b64ae40a7ca8d681 P 9bcc8a5781ff4eb5bcbe77690fd0c0d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6627f8faaf1423fb7504793451b18c3" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 44955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:48:07.063441 16671 mvcc.cc:205] Tried to move back new op lower bound from 7315834826998829056 to 7315834826940862464. Current Snapshot: MvccSnapshot[applied={T|T < 7315834826998829056}]
I20260807 08:48:07.063644 16673 mvcc.cc:205] Tried to move back new op lower bound from 7315834826998829056 to 7315834826940862464. Current Snapshot: MvccSnapshot[applied={T|T < 7315834826998829056}]
I20260807 08:48:07.065501 15852 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52334:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:48:07.065593 15852 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52334:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:48:07.066320 15852 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4ea7dd9dc02e498b932350ceaf4214cb: Sending DeleteTablet for 3 replicas of tablet bff437cdbea54a67b64ae40a7ca8d681
I20260807 08:48:07.066671 16068 tablet_service.cc:1514] Processing DeleteTablet for tablet bff437cdbea54a67b64ae40a7ca8d681 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:48:07 UTC) from {username='oryx', principal='oryx/127.4.159.190@KRBTEST.COM'} at 127.0.0.1:57072
I20260807 08:48:07.066686 16504 tablet_service.cc:1514] Processing DeleteTablet for tablet bff437cdbea54a67b64ae40a7ca8d681 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:48:07 UTC) from {username='oryx', principal='oryx/127.4.159.190@KRBTEST.COM'} at 127.0.0.1:41800
I20260807 08:48:07.066802 16286 tablet_service.cc:1514] Processing DeleteTablet for tablet bff437cdbea54a67b64ae40a7ca8d681 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:48:07 UTC) from {username='oryx', principal='oryx/127.4.159.190@KRBTEST.COM'} at 127.0.0.1:60196
I20260807 08:48:07.066924 16687 tablet_replica.cc:330] T bff437cdbea54a67b64ae40a7ca8d681 P 9bcc8a5781ff4eb5bcbe77690fd0c0d4: stopping tablet replica
I20260807 08:48:07.067009 16687 raft_consensus.cc:2229] T bff437cdbea54a67b64ae40a7ca8d681 P 9bcc8a5781ff4eb5bcbe77690fd0c0d4 [term 1 LEADER]: Raft consensus shutting down.
I20260807 08:48:07.067094 16687 raft_consensus.cc:2258] T bff437cdbea54a67b64ae40a7ca8d681 P 9bcc8a5781ff4eb5bcbe77690fd0c0d4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:48:07.067133 15852 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52334:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 08:48:07.067250 16693 tablet_replica.cc:330] T bff437cdbea54a67b64ae40a7ca8d681 P a6627f8faaf1423fb7504793451b18c3: stopping tablet replica
W20260807 08:48:07.067270 15852 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:48:07.067340 16693 raft_consensus.cc:2229] T bff437cdbea54a67b64ae40a7ca8d681 P a6627f8faaf1423fb7504793451b18c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:48:07.067440 16693 raft_consensus.cc:2258] T bff437cdbea54a67b64ae40a7ca8d681 P a6627f8faaf1423fb7504793451b18c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:48:07.067811 16687 ts_tablet_manager.cc:1905] T bff437cdbea54a67b64ae40a7ca8d681 P 9bcc8a5781ff4eb5bcbe77690fd0c0d4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:48:07.067811 16693 ts_tablet_manager.cc:1905] T bff437cdbea54a67b64ae40a7ca8d681 P a6627f8faaf1423fb7504793451b18c3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:48:07.068547 16504 tablet_service.cc:1467] Processing CreateTablet for tablet e18fe512fdfc481bb6784da895df04d1 (DEFAULT_TABLE table=test-table [id=d15595b603ac4993a40a24858ff88bd4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:48:07.068825 16504 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e18fe512fdfc481bb6784da895df04d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:48:07.069447 16693 ts_tablet_manager.cc:1918] T bff437cdbea54a67b64ae40a7ca8d681 P a6627f8faaf1423fb7504793451b18c3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:48:07.069459 16687 ts_tablet_manager.cc:1918] T bff437cdbea54a67b64ae40a7ca8d681 P 9bcc8a5781ff4eb5bcbe77690fd0c0d4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:48:07.069525 16693 log.cc:1196] T bff437cdbea54a67b64ae40a7ca8d681 P a6627f8faaf1423fb7504793451b18c3: Deleting WAL directory at /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786092453486433-4734-0/minicluster-data/ts-1/wal/wals/bff437cdbea54a67b64ae40a7ca8d681
I20260807 08:48:07.069633 16687 log.cc:1196] T bff437cdbea54a67b64ae40a7ca8d681 P 9bcc8a5781ff4eb5bcbe77690fd0c0d4: Deleting WAL directory at /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786092453486433-4734-0/minicluster-data/ts-2/wal/wals/bff437cdbea54a67b64ae40a7ca8d681
I20260807 08:48:07.069788 16693 ts_tablet_manager.cc:1939] T bff437cdbea54a67b64ae40a7ca8d681 P a6627f8faaf1423fb7504793451b18c3: Deleting consensus metadata
I20260807 08:48:07.069964 16687 ts_tablet_manager.cc:1939] T bff437cdbea54a67b64ae40a7ca8d681 P 9bcc8a5781ff4eb5bcbe77690fd0c0d4: Deleting consensus metadata
I20260807 08:48:07.070173 15824 catalog_manager.cc:4841] TS a6627f8faaf1423fb7504793451b18c3 (127.4.159.130:44955): tablet bff437cdbea54a67b64ae40a7ca8d681 (table test-table [id=557cde2f3f854de9a5d34b4ce525f7ef]) successfully deleted
I20260807 08:48:07.070243 16649 tablet_bootstrap.cc:492] T e18fe512fdfc481bb6784da895df04d1 P 9bcc8a5781ff4eb5bcbe77690fd0c0d4: Bootstrap starting.
I20260807 08:48:07.070685 16649 tablet_bootstrap.cc:654] T e18fe512fdfc481bb6784da895df04d1 P 9bcc8a5781ff4eb5bcbe77690fd0c0d4: Neither blocks nor log segments found. Creating new log.
I20260807 08:48:07.071116 15829 catalog_manager.cc:4841] TS 9bcc8a5781ff4eb5bcbe77690fd0c0d4 (127.4.159.131:42305): tablet bff437cdbea54a67b64ae40a7ca8d681 (table test-table [id=557cde2f3f854de9a5d34b4ce525f7ef]) successfully deleted
I20260807 08:48:07.071290 16649 tablet_bootstrap.cc:492] T e18fe512fdfc481bb6784da895df04d1 P 9bcc8a5781ff4eb5bcbe77690fd0c0d4: No bootstrap required, opened a new log
I20260807 08:48:07.071314 16286 tablet_service.cc:1467] Processing CreateTablet for tablet e18fe512fdfc481bb6784da895df04d1 (DEFAULT_TABLE table=test-table [id=d15595b603ac4993a40a24858ff88bd4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:48:07.071339 16649 ts_tablet_manager.cc:1397] T e18fe512fdfc481bb6784da895df04d1 P 9bcc8a5781ff4eb5bcbe77690fd0c0d4: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260807 08:48:07.071422 16286 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e18fe512fdfc481bb6784da895df04d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:48:07.071478 16649 raft_consensus.cc:348] T e18fe512fdfc481bb6784da895df04d1 P 9bcc8a5781ff4eb5bcbe77690fd0c0d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f8ad558036e4bcbbace591712a92519" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 41925 } } peers { permanent_uuid: "a6627f8faaf1423fb7504793451b18c3" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 44955 } } peers { permanent_uuid: "9bcc8a5781ff4eb5bcbe77690fd0c0d4" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 42305 } }
I20260807 08:48:07.071542 16649 raft_consensus.cc:374] T e18fe512fdfc481bb6784da895df04d1 P 9bcc8a5781ff4eb5bcbe77690fd0c0d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:48:07.071559 16649 raft_consensus.cc:729] T e18fe512fdfc481bb6784da895df04d1 P 9bcc8a5781ff4eb5bcbe77690fd0c0d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9bcc8a5781ff4eb5bcbe77690fd0c0d4, State: Initialized, Role: FOLLOWER
I20260807 08:48:07.071599 16649 consensus_queue.cc:260] T e18fe512fdfc481bb6784da895df04d1 P 9bcc8a5781ff4eb5bcbe77690fd0c0d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f8ad558036e4bcbbace591712a92519" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 41925 } } peers { permanent_uuid: "a6627f8faaf1423fb7504793451b18c3" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 44955 } } peers { permanent_uuid: "9bcc8a5781ff4eb5bcbe77690fd0c0d4" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 42305 } }
I20260807 08:48:07.071681 16649 ts_tablet_manager.cc:1428] T e18fe512fdfc481bb6784da895df04d1 P 9bcc8a5781ff4eb5bcbe77690fd0c0d4: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260807 08:48:07.072544 16648 tablet_bootstrap.cc:492] T e18fe512fdfc481bb6784da895df04d1 P a6627f8faaf1423fb7504793451b18c3: Bootstrap starting.
I20260807 08:48:07.072867 16068 tablet_service.cc:1467] Processing CreateTablet for tablet e18fe512fdfc481bb6784da895df04d1 (DEFAULT_TABLE table=test-table [id=d15595b603ac4993a40a24858ff88bd4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:48:07.072933 16648 tablet_bootstrap.cc:654] T e18fe512fdfc481bb6784da895df04d1 P a6627f8faaf1423fb7504793451b18c3: Neither blocks nor log segments found. Creating new log.
I20260807 08:48:07.072986 16068 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e18fe512fdfc481bb6784da895df04d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:48:07.073513 16648 tablet_bootstrap.cc:492] T e18fe512fdfc481bb6784da895df04d1 P a6627f8faaf1423fb7504793451b18c3: No bootstrap required, opened a new log
I20260807 08:48:07.073561 16648 ts_tablet_manager.cc:1397] T e18fe512fdfc481bb6784da895df04d1 P a6627f8faaf1423fb7504793451b18c3: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 08:48:07.073709 16648 raft_consensus.cc:348] T e18fe512fdfc481bb6784da895df04d1 P a6627f8faaf1423fb7504793451b18c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f8ad558036e4bcbbace591712a92519" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 41925 } } peers { permanent_uuid: "a6627f8faaf1423fb7504793451b18c3" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 44955 } } peers { permanent_uuid: "9bcc8a5781ff4eb5bcbe77690fd0c0d4" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 42305 } }
I20260807 08:48:07.073787 16648 raft_consensus.cc:374] T e18fe512fdfc481bb6784da895df04d1 P a6627f8faaf1423fb7504793451b18c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:48:07.073812 16648 raft_consensus.cc:729] T e18fe512fdfc481bb6784da895df04d1 P a6627f8faaf1423fb7504793451b18c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6627f8faaf1423fb7504793451b18c3, State: Initialized, Role: FOLLOWER
I20260807 08:48:07.073848 16648 consensus_queue.cc:260] T e18fe512fdfc481bb6784da895df04d1 P a6627f8faaf1423fb7504793451b18c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f8ad558036e4bcbbace591712a92519" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 41925 } } peers { permanent_uuid: "a6627f8faaf1423fb7504793451b18c3" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 44955 } } peers { permanent_uuid: "9bcc8a5781ff4eb5bcbe77690fd0c0d4" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 42305 } }
I20260807 08:48:07.073941 16648 ts_tablet_manager.cc:1428] T e18fe512fdfc481bb6784da895df04d1 P a6627f8faaf1423fb7504793451b18c3: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 08:48:07.074251 16653 tablet_bootstrap.cc:492] T e18fe512fdfc481bb6784da895df04d1 P 4f8ad558036e4bcbbace591712a92519: Bootstrap starting.
I20260807 08:48:07.074940 16653 tablet_bootstrap.cc:654] T e18fe512fdfc481bb6784da895df04d1 P 4f8ad558036e4bcbbace591712a92519: Neither blocks nor log segments found. Creating new log.
I20260807 08:48:07.075237 16688 tablet_replica.cc:330] T bff437cdbea54a67b64ae40a7ca8d681 P 4f8ad558036e4bcbbace591712a92519: stopping tablet replica
I20260807 08:48:07.075347 16688 raft_consensus.cc:2229] T bff437cdbea54a67b64ae40a7ca8d681 P 4f8ad558036e4bcbbace591712a92519 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:48:07.075459 16688 raft_consensus.cc:2258] T bff437cdbea54a67b64ae40a7ca8d681 P 4f8ad558036e4bcbbace591712a92519 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:48:07.075554 16653 tablet_bootstrap.cc:492] T e18fe512fdfc481bb6784da895df04d1 P 4f8ad558036e4bcbbace591712a92519: No bootstrap required, opened a new log
I20260807 08:48:07.075604 16653 ts_tablet_manager.cc:1397] T e18fe512fdfc481bb6784da895df04d1 P 4f8ad558036e4bcbbace591712a92519: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 08:48:07.075742 16653 raft_consensus.cc:348] T e18fe512fdfc481bb6784da895df04d1 P 4f8ad558036e4bcbbace591712a92519 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f8ad558036e4bcbbace591712a92519" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 41925 } } peers { permanent_uuid: "a6627f8faaf1423fb7504793451b18c3" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 44955 } } peers { permanent_uuid: "9bcc8a5781ff4eb5bcbe77690fd0c0d4" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 42305 } }
I20260807 08:48:07.075793 16653 raft_consensus.cc:374] T e18fe512fdfc481bb6784da895df04d1 P 4f8ad558036e4bcbbace591712a92519 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:48:07.075811 16653 raft_consensus.cc:729] T e18fe512fdfc481bb6784da895df04d1 P 4f8ad558036e4bcbbace591712a92519 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f8ad558036e4bcbbace591712a92519, State: Initialized, Role: FOLLOWER
I20260807 08:48:07.075850 16653 consensus_queue.cc:260] T e18fe512fdfc481bb6784da895df04d1 P 4f8ad558036e4bcbbace591712a92519 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f8ad558036e4bcbbace591712a92519" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 41925 } } peers { permanent_uuid: "a6627f8faaf1423fb7504793451b18c3" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 44955 } } peers { permanent_uuid: "9bcc8a5781ff4eb5bcbe77690fd0c0d4" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 42305 } }
I20260807 08:48:07.075935 16653 ts_tablet_manager.cc:1428] T e18fe512fdfc481bb6784da895df04d1 P 4f8ad558036e4bcbbace591712a92519: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 08:48:07.076452 16688 ts_tablet_manager.cc:1905] T bff437cdbea54a67b64ae40a7ca8d681 P 4f8ad558036e4bcbbace591712a92519: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:48:07.077899 16688 ts_tablet_manager.cc:1918] T bff437cdbea54a67b64ae40a7ca8d681 P 4f8ad558036e4bcbbace591712a92519: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:48:07.077975 16688 log.cc:1196] T bff437cdbea54a67b64ae40a7ca8d681 P 4f8ad558036e4bcbbace591712a92519: Deleting WAL directory at /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786092453486433-4734-0/minicluster-data/ts-0/wal/wals/bff437cdbea54a67b64ae40a7ca8d681
I20260807 08:48:07.078214 16688 ts_tablet_manager.cc:1939] T bff437cdbea54a67b64ae40a7ca8d681 P 4f8ad558036e4bcbbace591712a92519: Deleting consensus metadata
I20260807 08:48:07.078596 15826 catalog_manager.cc:4841] TS 4f8ad558036e4bcbbace591712a92519 (127.4.159.129:41925): tablet bff437cdbea54a67b64ae40a7ca8d681 (table test-table [id=557cde2f3f854de9a5d34b4ce525f7ef]) successfully deleted
W20260807 08:48:07.120071 16171 tablet.cc:2369] T e18fe512fdfc481bb6784da895df04d1 P 4f8ad558036e4bcbbace591712a92519: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 08:48:07.154291 16612 tablet.cc:2369] T e18fe512fdfc481bb6784da895df04d1 P 9bcc8a5781ff4eb5bcbe77690fd0c0d4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 08:48:07.183351 16664 raft_consensus.cc:482] T e18fe512fdfc481bb6784da895df04d1 P 4f8ad558036e4bcbbace591712a92519 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:48:07.183439 16664 raft_consensus.cc:504] T e18fe512fdfc481bb6784da895df04d1 P 4f8ad558036e4bcbbace591712a92519 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f8ad558036e4bcbbace591712a92519" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 41925 } } peers { permanent_uuid: "a6627f8faaf1423fb7504793451b18c3" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 44955 } } peers { permanent_uuid: "9bcc8a5781ff4eb5bcbe77690fd0c0d4" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 42305 } }
I20260807 08:48:07.183717 16664 leader_election.cc:288] T e18fe512fdfc481bb6784da895df04d1 P 4f8ad558036e4bcbbace591712a92519 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a6627f8faaf1423fb7504793451b18c3 (127.4.159.130:44955), 9bcc8a5781ff4eb5bcbe77690fd0c0d4 (127.4.159.131:42305)
I20260807 08:48:07.187472 16646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:07.183938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.129:41529 (local address 127.4.159.131:42305)
0807 08:48:07.183983 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:07.183985 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:07.184211 (+ 226us) server_negotiation.cc:407] Connection header received
0807 08:48:07.184243 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:07.184244 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:07.184254 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:07.184302 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:48:07.184467 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:07.185147 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:07.186588 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:07.187130 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:07.187199 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:07.187208 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 08:48:07.187369 (+ 161us) server_negotiation.cc:299] Negotiation successful
0807 08:48:07.187403 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 08:48:07.187472 16645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:07.183939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.129:46301 (local address 127.4.159.130:44955)
0807 08:48:07.183983 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:07.183985 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:07.184113 (+ 128us) server_negotiation.cc:407] Connection header received
0807 08:48:07.184169 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:07.184170 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:07.184180 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:07.184235 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:48:07.184371 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:07.185111 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:07.186590 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:07.187130 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:07.187199 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:07.187207 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0807 08:48:07.187370 (+ 163us) server_negotiation.cc:299] Negotiation successful
0807 08:48:07.187404 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260807 08:48:07.187515 16698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:07.183886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.131:42305 (local address 127.4.159.129:41529)
0807 08:48:07.184126 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:07.184135 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:07.184194 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:07.184310 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:07.184310 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:07.184313 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:48:07.184394 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:07.184396 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:07.185218 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:07.185219 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:07.186513 (+ 1294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:07.186517 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:07.187217 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:07.187218 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:07.187301 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:07.187306 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 08:48:07.187358 (+ 52us) client_negotiation.cc:240] Negotiation successful
0807 08:48:07.187409 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":228,"mutex_wait_us":61,"thread_start_us":54,"threads_started":1}
I20260807 08:48:07.187515 16700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:07.183884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.130:44955 (local address 127.4.159.129:46301)
0807 08:48:07.184038 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:07.184047 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:07.184112 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:07.184239 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:07.184240 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:07.184242 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:48:07.184312 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:07.184314 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:07.185196 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:07.185197 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:07.186514 (+ 1317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:07.186517 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:07.187215 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:07.187217 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:07.187301 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:07.187307 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 08:48:07.187358 (+ 51us) client_negotiation.cc:240] Negotiation successful
0807 08:48:07.187409 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"mutex_wait_us":52,"thread_start_us":71,"threads_started":1}
I20260807 08:48:07.187757 16534 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e18fe512fdfc481bb6784da895df04d1" candidate_uuid: "4f8ad558036e4bcbbace591712a92519" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9bcc8a5781ff4eb5bcbe77690fd0c0d4" is_pre_election: true
I20260807 08:48:07.187891 16534 raft_consensus.cc:2454] T e18fe512fdfc481bb6784da895df04d1 P 9bcc8a5781ff4eb5bcbe77690fd0c0d4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f8ad558036e4bcbbace591712a92519 in term 0.
I20260807 08:48:07.187909 16317 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e18fe512fdfc481bb6784da895df04d1" candidate_uuid: "4f8ad558036e4bcbbace591712a92519" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6627f8faaf1423fb7504793451b18c3" is_pre_election: true
I20260807 08:48:07.188004 16317 raft_consensus.cc:2454] T e18fe512fdfc481bb6784da895df04d1 P a6627f8faaf1423fb7504793451b18c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f8ad558036e4bcbbace591712a92519 in term 0.
I20260807 08:48:07.188073 15999 leader_election.cc:302] T e18fe512fdfc481bb6784da895df04d1 P 4f8ad558036e4bcbbace591712a92519 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f8ad558036e4bcbbace591712a92519, 9bcc8a5781ff4eb5bcbe77690fd0c0d4; no voters:
I20260807 08:48:07.188203 16664 raft_consensus.cc:2781] T e18fe512fdfc481bb6784da895df04d1 P 4f8ad558036e4bcbbace591712a92519 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:48:07.188251 16664 raft_consensus.cc:482] T e18fe512fdfc481bb6784da895df04d1 P 4f8ad558036e4bcbbace591712a92519 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:48:07.188270 16664 raft_consensus.cc:3037] T e18fe512fdfc481bb6784da895df04d1 P 4f8ad558036e4bcbbace591712a92519 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:48:07.189049 16664 raft_consensus.cc:504] T e18fe512fdfc481bb6784da895df04d1 P 4f8ad558036e4bcbbace591712a92519 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f8ad558036e4bcbbace591712a92519" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 41925 } } peers { permanent_uuid: "a6627f8faaf1423fb7504793451b18c3" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 44955 } } peers { permanent_uuid: "9bcc8a5781ff4eb5bcbe77690fd0c0d4" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 42305 } }
I20260807 08:48:07.189178 16664 leader_election.cc:288] T e18fe512fdfc481bb6784da895df04d1 P 4f8ad558036e4bcbbace591712a92519 [CANDIDATE]: Term 1 election: Requested vote from peers a6627f8faaf1423fb7504793451b18c3 (127.4.159.130:44955), 9bcc8a5781ff4eb5bcbe77690fd0c0d4 (127.4.159.131:42305)
I20260807 08:48:07.189327 16317 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e18fe512fdfc481bb6784da895df04d1" candidate_uuid: "4f8ad558036e4bcbbace591712a92519" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6627f8faaf1423fb7504793451b18c3"
I20260807 08:48:07.189332 16534 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e18fe512fdfc481bb6784da895df04d1" candidate_uuid: "4f8ad558036e4bcbbace591712a92519" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9bcc8a5781ff4eb5bcbe77690fd0c0d4"
I20260807 08:48:07.189406 16534 raft_consensus.cc:3037] T e18fe512fdfc481bb6784da895df04d1 P 9bcc8a5781ff4eb5bcbe77690fd0c0d4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:48:07.189400 16317 raft_consensus.cc:3037] T e18fe512fdfc481bb6784da895df04d1 P a6627f8faaf1423fb7504793451b18c3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:48:07.189981 16317 raft_consensus.cc:2454] T e18fe512fdfc481bb6784da895df04d1 P a6627f8faaf1423fb7504793451b18c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f8ad558036e4bcbbace591712a92519 in term 1.
I20260807 08:48:07.190048 16534 raft_consensus.cc:2454] T e18fe512fdfc481bb6784da895df04d1 P 9bcc8a5781ff4eb5bcbe77690fd0c0d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f8ad558036e4bcbbace591712a92519 in term 1.
I20260807 08:48:07.190201 15994 leader_election.cc:302] T e18fe512fdfc481bb6784da895df04d1 P 4f8ad558036e4bcbbace591712a92519 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f8ad558036e4bcbbace591712a92519, a6627f8faaf1423fb7504793451b18c3; no voters:
I20260807 08:48:07.190312 16664 raft_consensus.cc:2781] T e18fe512fdfc481bb6784da895df04d1 P 4f8ad558036e4bcbbace591712a92519 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:48:07.190465 16664 raft_consensus.cc:686] T e18fe512fdfc481bb6784da895df04d1 P 4f8ad558036e4bcbbace591712a92519 [term 1 LEADER]: Becoming Leader. State: Replica: 4f8ad558036e4bcbbace591712a92519, State: Running, Role: LEADER
I20260807 08:48:07.190533 16664 consensus_queue.cc:237] T e18fe512fdfc481bb6784da895df04d1 P 4f8ad558036e4bcbbace591712a92519 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f8ad558036e4bcbbace591712a92519" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 41925 } } peers { permanent_uuid: "a6627f8faaf1423fb7504793451b18c3" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 44955 } } peers { permanent_uuid: "9bcc8a5781ff4eb5bcbe77690fd0c0d4" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 42305 } }
I20260807 08:48:07.191133 15852 catalog_manager.cc:5495] T e18fe512fdfc481bb6784da895df04d1 P 4f8ad558036e4bcbbace591712a92519 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4f8ad558036e4bcbbace591712a92519 (127.4.159.129). New cstate: current_term: 1 leader_uuid: "4f8ad558036e4bcbbace591712a92519" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f8ad558036e4bcbbace591712a92519" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 41925 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a6627f8faaf1423fb7504793451b18c3" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 44955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9bcc8a5781ff4eb5bcbe77690fd0c0d4" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 42305 } health_report { overall_health: UNKNOWN } } }
Aug 07 08:48:07 dist-test-slave-6zbq krb5kdc[15775](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092486, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.159.190@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM
I20260807 08:48:07.244671 16713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:07.238922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:45303 (local address 127.0.0.1:52350)
0807 08:48:07.239056 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:07.239065 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:07.239123 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:07.239229 (+ 106us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:07.239229 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:07.239503 (+ 274us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:48:07.239548 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:07.239550 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:07.240127 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:07.240128 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:07.240665 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:07.240667 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:07.240928 (+ 261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:07.240929 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:07.240974 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:07.241386 (+ 412us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:48:07.241391 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:48:07.242656 (+ 1265us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:48:07.243859 (+ 1203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:48:07.243866 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:48:07.243868 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:48:07.244052 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:48:07.244204 (+ 152us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:48:07.244205 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:48:07.244205 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:48:07.244255 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:48:07.244416 (+ 161us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:48:07.244420 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:48:07.244455 (+ 35us) client_negotiation.cc:769] Sending connection context
0807 08:48:07.244502 (+ 47us) client_negotiation.cc:240] Negotiation successful
0807 08:48:07.244583 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":83,"threads_started":1}
I20260807 08:48:07.244724 15948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:07.238964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52350 (local address 127.4.159.190:45303)
0807 08:48:07.239018 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:07.239021 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:07.239116 (+ 95us) server_negotiation.cc:407] Connection header received
0807 08:48:07.239162 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:07.239163 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:07.239171 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:07.239212 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:07.239598 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:07.240075 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:07.240721 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:07.240886 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:07.240927 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:07.242736 (+ 1809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:48:07.242737 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:48:07.242738 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:48:07.242743 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:48:07.243798 (+ 1055us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:48:07.244119 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:48:07.244119 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:48:07.244120 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:48:07.244163 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:48:07.244292 (+ 129us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:48:07.244293 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:48:07.244293 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:48:07.244379 (+ 86us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:48:07.244421 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0807 08:48:07.244524 (+ 103us) server_negotiation.cc:299] Negotiation successful
0807 08:48:07.244629 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":45,"server-negotiator.queue_time_us":30}
I20260807 08:48:07.246423 15852 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.4.159.190@KRBTEST.COM'} at 127.0.0.1:52350:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 08:48:07.246610 15852 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:48:07.247890 16068 tablet_service.cc:1467] Processing CreateTablet for tablet 656cab67ce7c4b229f08452085fc3bee (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e15e580a014c47269081cbc544adf10e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 08:48:07.247906 16286 tablet_service.cc:1467] Processing CreateTablet for tablet 656cab67ce7c4b229f08452085fc3bee (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e15e580a014c47269081cbc544adf10e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 08:48:07.247920 16504 tablet_service.cc:1467] Processing CreateTablet for tablet 656cab67ce7c4b229f08452085fc3bee (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e15e580a014c47269081cbc544adf10e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 08:48:07.248045 16504 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 656cab67ce7c4b229f08452085fc3bee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:48:07.248046 16068 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 656cab67ce7c4b229f08452085fc3bee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:48:07.248046 16286 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 656cab67ce7c4b229f08452085fc3bee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:48:07.249100 16648 tablet_bootstrap.cc:492] T 656cab67ce7c4b229f08452085fc3bee P a6627f8faaf1423fb7504793451b18c3: Bootstrap starting.
I20260807 08:48:07.249467 16649 tablet_bootstrap.cc:492] T 656cab67ce7c4b229f08452085fc3bee P 9bcc8a5781ff4eb5bcbe77690fd0c0d4: Bootstrap starting.
I20260807 08:48:07.249526 16648 tablet_bootstrap.cc:654] T 656cab67ce7c4b229f08452085fc3bee P a6627f8faaf1423fb7504793451b18c3: Neither blocks nor log segments found. Creating new log.
I20260807 08:48:07.249781 16649 tablet_bootstrap.cc:654] T 656cab67ce7c4b229f08452085fc3bee P 9bcc8a5781ff4eb5bcbe77690fd0c0d4: Neither blocks nor log segments found. Creating new log.
I20260807 08:48:07.250018 16653 tablet_bootstrap.cc:492] T 656cab67ce7c4b229f08452085fc3bee P 4f8ad558036e4bcbbace591712a92519: Bootstrap starting.
I20260807 08:48:07.250156 16648 tablet_bootstrap.cc:492] T 656cab67ce7c4b229f08452085fc3bee P a6627f8faaf1423fb7504793451b18c3: No bootstrap required, opened a new log
I20260807 08:48:07.250207 16648 ts_tablet_manager.cc:1397] T 656cab67ce7c4b229f08452085fc3bee P a6627f8faaf1423fb7504793451b18c3: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 08:48:07.250322 16649 tablet_bootstrap.cc:492] T 656cab67ce7c4b229f08452085fc3bee P 9bcc8a5781ff4eb5bcbe77690fd0c0d4: No bootstrap required, opened a new log
I20260807 08:48:07.250386 16649 ts_tablet_manager.cc:1397] T 656cab67ce7c4b229f08452085fc3bee P 9bcc8a5781ff4eb5bcbe77690fd0c0d4: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260807 08:48:07.250419 16653 tablet_bootstrap.cc:654] T 656cab67ce7c4b229f08452085fc3bee P 4f8ad558036e4bcbbace591712a92519: Neither blocks nor log segments found. Creating new log.
I20260807 08:48:07.250392 16648 raft_consensus.cc:348] T 656cab67ce7c4b229f08452085fc3bee P a6627f8faaf1423fb7504793451b18c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f8ad558036e4bcbbace591712a92519" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 41925 } } peers { permanent_uuid: "a6627f8faaf1423fb7504793451b18c3" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 44955 } } peers { permanent_uuid: "9bcc8a5781ff4eb5bcbe77690fd0c0d4" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 42305 } }
I20260807 08:48:07.250460 16648 raft_consensus.cc:374] T 656cab67ce7c4b229f08452085fc3bee P a6627f8faaf1423fb7504793451b18c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:48:07.250478 16648 raft_consensus.cc:729] T 656cab67ce7c4b229f08452085fc3bee P a6627f8faaf1423fb7504793451b18c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6627f8faaf1423fb7504793451b18c3, State: Initialized, Role: FOLLOWER
I20260807 08:48:07.250494 16649 raft_consensus.cc:348] T 656cab67ce7c4b229f08452085fc3bee P 9bcc8a5781ff4eb5bcbe77690fd0c0d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f8ad558036e4bcbbace591712a92519" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 41925 } } peers { permanent_uuid: "a6627f8faaf1423fb7504793451b18c3" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 44955 } } peers { permanent_uuid: "9bcc8a5781ff4eb5bcbe77690fd0c0d4" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 42305 } }
I20260807 08:48:07.250548 16649 raft_consensus.cc:374] T 656cab67ce7c4b229f08452085fc3bee P 9bcc8a5781ff4eb5bcbe77690fd0c0d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:48:07.250524 16648 consensus_queue.cc:260] T 656cab67ce7c4b229f08452085fc3bee P a6627f8faaf1423fb7504793451b18c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f8ad558036e4bcbbace591712a92519" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 41925 } } peers { permanent_uuid: "a6627f8faaf1423fb7504793451b18c3" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 44955 } } peers { permanent_uuid: "9bcc8a5781ff4eb5bcbe77690fd0c0d4" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 42305 } }
I20260807 08:48:07.250568 16649 raft_consensus.cc:729] T 656cab67ce7c4b229f08452085fc3bee P 9bcc8a5781ff4eb5bcbe77690fd0c0d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9bcc8a5781ff4eb5bcbe77690fd0c0d4, State: Initialized, Role: FOLLOWER
I20260807 08:48:07.250604 16649 consensus_queue.cc:260] T 656cab67ce7c4b229f08452085fc3bee P 9bcc8a5781ff4eb5bcbe77690fd0c0d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f8ad558036e4bcbbace591712a92519" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 41925 } } peers { permanent_uuid: "a6627f8faaf1423fb7504793451b18c3" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 44955 } } peers { permanent_uuid: "9bcc8a5781ff4eb5bcbe77690fd0c0d4" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 42305 } }
I20260807 08:48:07.250629 16648 ts_tablet_manager.cc:1428] T 656cab67ce7c4b229f08452085fc3bee P a6627f8faaf1423fb7504793451b18c3: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 08:48:07.250687 16649 ts_tablet_manager.cc:1428] T 656cab67ce7c4b229f08452085fc3bee P 9bcc8a5781ff4eb5bcbe77690fd0c0d4: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 08:48:07.250713 16654 tablet_replica.cc:439] T 656cab67ce7c4b229f08452085fc3bee P a6627f8faaf1423fb7504793451b18c3: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f8ad558036e4bcbbace591712a92519" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 41925 } } peers { permanent_uuid: "a6627f8faaf1423fb7504793451b18c3" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 44955 } } peers { permanent_uuid: "9bcc8a5781ff4eb5bcbe77690fd0c0d4" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 42305 } } }
I20260807 08:48:07.250774 16679 tablet_replica.cc:439] T 656cab67ce7c4b229f08452085fc3bee P 9bcc8a5781ff4eb5bcbe77690fd0c0d4: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f8ad558036e4bcbbace591712a92519" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 41925 } } peers { permanent_uuid: "a6627f8faaf1423fb7504793451b18c3" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 44955 } } peers { permanent_uuid: "9bcc8a5781ff4eb5bcbe77690fd0c0d4" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 42305 } } }
I20260807 08:48:07.250803 16654 tablet_replica.cc:442] T 656cab67ce7c4b229f08452085fc3bee P a6627f8faaf1423fb7504793451b18c3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 08:48:07.250837 16679 tablet_replica.cc:442] T 656cab67ce7c4b229f08452085fc3bee P 9bcc8a5781ff4eb5bcbe77690fd0c0d4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 08:48:07.251267 16653 tablet_bootstrap.cc:492] T 656cab67ce7c4b229f08452085fc3bee P 4f8ad558036e4bcbbace591712a92519: No bootstrap required, opened a new log
I20260807 08:48:07.251336 16653 ts_tablet_manager.cc:1397] T 656cab67ce7c4b229f08452085fc3bee P 4f8ad558036e4bcbbace591712a92519: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260807 08:48:07.251480 16653 raft_consensus.cc:348] T 656cab67ce7c4b229f08452085fc3bee P 4f8ad558036e4bcbbace591712a92519 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f8ad558036e4bcbbace591712a92519" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 41925 } } peers { permanent_uuid: "a6627f8faaf1423fb7504793451b18c3" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 44955 } } peers { permanent_uuid: "9bcc8a5781ff4eb5bcbe77690fd0c0d4" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 42305 } }
I20260807 08:48:07.251547 16653 raft_consensus.cc:374] T 656cab67ce7c4b229f08452085fc3bee P 4f8ad558036e4bcbbace591712a92519 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:48:07.251565 16653 raft_consensus.cc:729] T 656cab67ce7c4b229f08452085fc3bee P 4f8ad558036e4bcbbace591712a92519 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f8ad558036e4bcbbace591712a92519, State: Initialized, Role: FOLLOWER
I20260807 08:48:07.251607 16653 consensus_queue.cc:260] T 656cab67ce7c4b229f08452085fc3bee P 4f8ad558036e4bcbbace591712a92519 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f8ad558036e4bcbbace591712a92519" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 41925 } } peers { permanent_uuid: "a6627f8faaf1423fb7504793451b18c3" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 44955 } } peers { permanent_uuid: "9bcc8a5781ff4eb5bcbe77690fd0c0d4" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 42305 } }
I20260807 08:48:07.251695 16653 ts_tablet_manager.cc:1428] T 656cab67ce7c4b229f08452085fc3bee P 4f8ad558036e4bcbbace591712a92519: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 08:48:07.251744 16664 tablet_replica.cc:439] T 656cab67ce7c4b229f08452085fc3bee P 4f8ad558036e4bcbbace591712a92519: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f8ad558036e4bcbbace591712a92519" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 41925 } } peers { permanent_uuid: "a6627f8faaf1423fb7504793451b18c3" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 44955 } } peers { permanent_uuid: "9bcc8a5781ff4eb5bcbe77690fd0c0d4" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 42305 } } }
I20260807 08:48:07.251835 16664 tablet_replica.cc:442] T 656cab67ce7c4b229f08452085fc3bee P 4f8ad558036e4bcbbace591712a92519: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 08:48:07.425187 16664 raft_consensus.cc:482] T 656cab67ce7c4b229f08452085fc3bee P 4f8ad558036e4bcbbace591712a92519 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:48:07.425275 16664 raft_consensus.cc:504] T 656cab67ce7c4b229f08452085fc3bee P 4f8ad558036e4bcbbace591712a92519 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f8ad558036e4bcbbace591712a92519" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 41925 } } peers { permanent_uuid: "a6627f8faaf1423fb7504793451b18c3" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 44955 } } peers { permanent_uuid: "9bcc8a5781ff4eb5bcbe77690fd0c0d4" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 42305 } }
I20260807 08:48:07.425428 16664 leader_election.cc:288] T 656cab67ce7c4b229f08452085fc3bee P 4f8ad558036e4bcbbace591712a92519 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a6627f8faaf1423fb7504793451b18c3 (127.4.159.130:44955), 9bcc8a5781ff4eb5bcbe77690fd0c0d4 (127.4.159.131:42305)
I20260807 08:48:07.425613 16534 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "656cab67ce7c4b229f08452085fc3bee" candidate_uuid: "4f8ad558036e4bcbbace591712a92519" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9bcc8a5781ff4eb5bcbe77690fd0c0d4" is_pre_election: true
I20260807 08:48:07.425621 16317 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "656cab67ce7c4b229f08452085fc3bee" candidate_uuid: "4f8ad558036e4bcbbace591712a92519" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6627f8faaf1423fb7504793451b18c3" is_pre_election: true
I20260807 08:48:07.425707 16534 raft_consensus.cc:2454] T 656cab67ce7c4b229f08452085fc3bee P 9bcc8a5781ff4eb5bcbe77690fd0c0d4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f8ad558036e4bcbbace591712a92519 in term 0.
I20260807 08:48:07.425714 16317 raft_consensus.cc:2454] T 656cab67ce7c4b229f08452085fc3bee P a6627f8faaf1423fb7504793451b18c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f8ad558036e4bcbbace591712a92519 in term 0.
I20260807 08:48:07.425851 15999 leader_election.cc:302] T 656cab67ce7c4b229f08452085fc3bee P 4f8ad558036e4bcbbace591712a92519 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f8ad558036e4bcbbace591712a92519, 9bcc8a5781ff4eb5bcbe77690fd0c0d4; no voters:
I20260807 08:48:07.425967 16664 raft_consensus.cc:2781] T 656cab67ce7c4b229f08452085fc3bee P 4f8ad558036e4bcbbace591712a92519 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:48:07.426013 16664 raft_consensus.cc:482] T 656cab67ce7c4b229f08452085fc3bee P 4f8ad558036e4bcbbace591712a92519 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:48:07.426035 16664 raft_consensus.cc:3037] T 656cab67ce7c4b229f08452085fc3bee P 4f8ad558036e4bcbbace591712a92519 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:48:07.426741 16664 raft_consensus.cc:504] T 656cab67ce7c4b229f08452085fc3bee P 4f8ad558036e4bcbbace591712a92519 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f8ad558036e4bcbbace591712a92519" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 41925 } } peers { permanent_uuid: "a6627f8faaf1423fb7504793451b18c3" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 44955 } } peers { permanent_uuid: "9bcc8a5781ff4eb5bcbe77690fd0c0d4" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 42305 } }
I20260807 08:48:07.426880 16664 leader_election.cc:288] T 656cab67ce7c4b229f08452085fc3bee P 4f8ad558036e4bcbbace591712a92519 [CANDIDATE]: Term 1 election: Requested vote from peers a6627f8faaf1423fb7504793451b18c3 (127.4.159.130:44955), 9bcc8a5781ff4eb5bcbe77690fd0c0d4 (127.4.159.131:42305)
I20260807 08:48:07.427022 16317 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "656cab67ce7c4b229f08452085fc3bee" candidate_uuid: "4f8ad558036e4bcbbace591712a92519" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6627f8faaf1423fb7504793451b18c3"
I20260807 08:48:07.427035 16534 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "656cab67ce7c4b229f08452085fc3bee" candidate_uuid: "4f8ad558036e4bcbbace591712a92519" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9bcc8a5781ff4eb5bcbe77690fd0c0d4"
I20260807 08:48:07.427095 16534 raft_consensus.cc:3037] T 656cab67ce7c4b229f08452085fc3bee P 9bcc8a5781ff4eb5bcbe77690fd0c0d4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:48:07.427093 16317 raft_consensus.cc:3037] T 656cab67ce7c4b229f08452085fc3bee P a6627f8faaf1423fb7504793451b18c3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:48:07.427793 16534 raft_consensus.cc:2454] T 656cab67ce7c4b229f08452085fc3bee P 9bcc8a5781ff4eb5bcbe77690fd0c0d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f8ad558036e4bcbbace591712a92519 in term 1.
I20260807 08:48:07.427805 16317 raft_consensus.cc:2454] T 656cab67ce7c4b229f08452085fc3bee P a6627f8faaf1423fb7504793451b18c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f8ad558036e4bcbbace591712a92519 in term 1.
I20260807 08:48:07.427959 15999 leader_election.cc:302] T 656cab67ce7c4b229f08452085fc3bee P 4f8ad558036e4bcbbace591712a92519 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f8ad558036e4bcbbace591712a92519, 9bcc8a5781ff4eb5bcbe77690fd0c0d4; no voters:
I20260807 08:48:07.428061 16664 raft_consensus.cc:2781] T 656cab67ce7c4b229f08452085fc3bee P 4f8ad558036e4bcbbace591712a92519 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:48:07.428117 16664 raft_consensus.cc:686] T 656cab67ce7c4b229f08452085fc3bee P 4f8ad558036e4bcbbace591712a92519 [term 1 LEADER]: Becoming Leader. State: Replica: 4f8ad558036e4bcbbace591712a92519, State: Running, Role: LEADER
I20260807 08:48:07.428159 16664 consensus_queue.cc:237] T 656cab67ce7c4b229f08452085fc3bee P 4f8ad558036e4bcbbace591712a92519 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f8ad558036e4bcbbace591712a92519" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 41925 } } peers { permanent_uuid: "a6627f8faaf1423fb7504793451b18c3" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 44955 } } peers { permanent_uuid: "9bcc8a5781ff4eb5bcbe77690fd0c0d4" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 42305 } }
I20260807 08:48:07.428455 16701 tablet_replica.cc:439] T 656cab67ce7c4b229f08452085fc3bee P 4f8ad558036e4bcbbace591712a92519: TxnStatusTablet state changed. Reason: New leader 4f8ad558036e4bcbbace591712a92519. Latest consensus state: current_term: 1 leader_uuid: "4f8ad558036e4bcbbace591712a92519" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f8ad558036e4bcbbace591712a92519" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 41925 } } peers { permanent_uuid: "a6627f8faaf1423fb7504793451b18c3" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 44955 } } peers { permanent_uuid: "9bcc8a5781ff4eb5bcbe77690fd0c0d4" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 42305 } } }
I20260807 08:48:07.428560 16701 tablet_replica.cc:442] T 656cab67ce7c4b229f08452085fc3bee P 4f8ad558036e4bcbbace591712a92519: This TxnStatusTablet replica's current role is: LEADER
I20260807 08:48:07.428750 16728 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 08:48:07.428858 16728 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 08:48:07.428794 15852 catalog_manager.cc:5495] T 656cab67ce7c4b229f08452085fc3bee P 4f8ad558036e4bcbbace591712a92519 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4f8ad558036e4bcbbace591712a92519 (127.4.159.129). New cstate: current_term: 1 leader_uuid: "4f8ad558036e4bcbbace591712a92519" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f8ad558036e4bcbbace591712a92519" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 41925 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a6627f8faaf1423fb7504793451b18c3" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 44955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9bcc8a5781ff4eb5bcbe77690fd0c0d4" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 42305 } health_report { overall_health: UNKNOWN } } }
I20260807 08:48:07.476831 16643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:07.474173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57080 (local address 127.4.159.129:41925)
0807 08:48:07.474236 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:07.474238 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:07.474248 (+ 10us) server_negotiation.cc:407] Connection header received
0807 08:48:07.474312 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:07.474313 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:07.474332 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:07.474417 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:48:07.474514 (+ 97us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:07.475258 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:07.475976 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:07.476292 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:07.476358 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:07.476527 (+ 169us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:48:07.476641 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:48:07.476703 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 08:48:07.476736 (+ 33us) server_negotiation.cc:299] Negotiation successful
0807 08:48:07.476756 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260807 08:48:07.476840 16713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:07.474103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.129:41925 (local address 127.0.0.1:57080)
0807 08:48:07.474172 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:07.474184 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:07.474235 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:07.474423 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:07.474424 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:07.474425 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 08:48:07.474469 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:07.474472 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:07.475339 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:07.475339 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:07.475896 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:07.475897 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:07.476375 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:07.476376 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:07.476431 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:07.476447 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 08:48:07.476697 (+ 250us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 08:48:07.476700 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 08:48:07.476728 (+ 28us) client_negotiation.cc:240] Negotiation successful
0807 08:48:07.476768 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260807 08:48:07.477849 16317 raft_consensus.cc:1261] T 656cab67ce7c4b229f08452085fc3bee P a6627f8faaf1423fb7504793451b18c3 [term 1 FOLLOWER]: Refusing update from remote peer 4f8ad558036e4bcbbace591712a92519: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:48:07.477900 16534 raft_consensus.cc:1261] T 656cab67ce7c4b229f08452085fc3bee P 9bcc8a5781ff4eb5bcbe77690fd0c0d4 [term 1 FOLLOWER]: Refusing update from remote peer 4f8ad558036e4bcbbace591712a92519: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:48:07.478094 16664 consensus_queue.cc:1035] T 656cab67ce7c4b229f08452085fc3bee P 4f8ad558036e4bcbbace591712a92519 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6627f8faaf1423fb7504793451b18c3" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 44955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:48:07.478221 16701 consensus_queue.cc:1035] T 656cab67ce7c4b229f08452085fc3bee P 4f8ad558036e4bcbbace591712a92519 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9bcc8a5781ff4eb5bcbe77690fd0c0d4" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 42305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:48:07.478804 16654 tablet_replica.cc:439] T 656cab67ce7c4b229f08452085fc3bee P a6627f8faaf1423fb7504793451b18c3: TxnStatusTablet state changed. Reason: New leader 4f8ad558036e4bcbbace591712a92519. Latest consensus state: current_term: 1 leader_uuid: "4f8ad558036e4bcbbace591712a92519" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f8ad558036e4bcbbace591712a92519" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 41925 } } peers { permanent_uuid: "a6627f8faaf1423fb7504793451b18c3" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 44955 } } peers { permanent_uuid: "9bcc8a5781ff4eb5bcbe77690fd0c0d4" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 42305 } } }
I20260807 08:48:07.478890 16654 tablet_replica.cc:442] T 656cab67ce7c4b229f08452085fc3bee P a6627f8faaf1423fb7504793451b18c3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 08:48:07.479019 16679 tablet_replica.cc:439] T 656cab67ce7c4b229f08452085fc3bee P 9bcc8a5781ff4eb5bcbe77690fd0c0d4: TxnStatusTablet state changed. Reason: New leader 4f8ad558036e4bcbbace591712a92519. Latest consensus state: current_term: 1 leader_uuid: "4f8ad558036e4bcbbace591712a92519" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f8ad558036e4bcbbace591712a92519" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 41925 } } peers { permanent_uuid: "a6627f8faaf1423fb7504793451b18c3" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 44955 } } peers { permanent_uuid: "9bcc8a5781ff4eb5bcbe77690fd0c0d4" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 42305 } } }
I20260807 08:48:07.479103 16679 tablet_replica.cc:442] T 656cab67ce7c4b229f08452085fc3bee P 9bcc8a5781ff4eb5bcbe77690fd0c0d4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 08:48:07.479461 16675 mvcc.cc:205] Tried to move back new op lower bound from 7315834828707418112 to 7315834828506095616. Current Snapshot: MvccSnapshot[applied={T|T < 7315834828707418112}]
I20260807 08:48:07.479664 16654 tablet_replica.cc:439] T 656cab67ce7c4b229f08452085fc3bee P a6627f8faaf1423fb7504793451b18c3: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4f8ad558036e4bcbbace591712a92519" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f8ad558036e4bcbbace591712a92519" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 41925 } } peers { permanent_uuid: "a6627f8faaf1423fb7504793451b18c3" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 44955 } } peers { permanent_uuid: "9bcc8a5781ff4eb5bcbe77690fd0c0d4" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 42305 } } }
I20260807 08:48:07.479738 16654 tablet_replica.cc:442] T 656cab67ce7c4b229f08452085fc3bee P a6627f8faaf1423fb7504793451b18c3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 08:48:07.479750 16701 tablet_replica.cc:439] T 656cab67ce7c4b229f08452085fc3bee P 4f8ad558036e4bcbbace591712a92519: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4f8ad558036e4bcbbace591712a92519" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f8ad558036e4bcbbace591712a92519" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 41925 } } peers { permanent_uuid: "a6627f8faaf1423fb7504793451b18c3" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 44955 } } peers { permanent_uuid: "9bcc8a5781ff4eb5bcbe77690fd0c0d4" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 42305 } } }
I20260807 08:48:07.479818 16701 tablet_replica.cc:442] T 656cab67ce7c4b229f08452085fc3bee P 4f8ad558036e4bcbbace591712a92519: This TxnStatusTablet replica's current role is: LEADER
I20260807 08:48:07.479846 16664 tablet_replica.cc:439] T 656cab67ce7c4b229f08452085fc3bee P 4f8ad558036e4bcbbace591712a92519: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4f8ad558036e4bcbbace591712a92519" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f8ad558036e4bcbbace591712a92519" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 41925 } } peers { permanent_uuid: "a6627f8faaf1423fb7504793451b18c3" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 44955 } } peers { permanent_uuid: "9bcc8a5781ff4eb5bcbe77690fd0c0d4" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 42305 } } }
I20260807 08:48:07.479907 16664 tablet_replica.cc:442] T 656cab67ce7c4b229f08452085fc3bee P 4f8ad558036e4bcbbace591712a92519: This TxnStatusTablet replica's current role is: LEADER
I20260807 08:48:07.480698 16679 tablet_replica.cc:439] T 656cab67ce7c4b229f08452085fc3bee P 9bcc8a5781ff4eb5bcbe77690fd0c0d4: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4f8ad558036e4bcbbace591712a92519" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f8ad558036e4bcbbace591712a92519" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 41925 } } peers { permanent_uuid: "a6627f8faaf1423fb7504793451b18c3" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 44955 } } peers { permanent_uuid: "9bcc8a5781ff4eb5bcbe77690fd0c0d4" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 42305 } } }
I20260807 08:48:07.480787 16679 tablet_replica.cc:442] T 656cab67ce7c4b229f08452085fc3bee P 9bcc8a5781ff4eb5bcbe77690fd0c0d4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 08:48:07.483492 16643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:07.481334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57096 (local address 127.4.159.129:41925)
0807 08:48:07.481367 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:07.481369 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:07.481377 (+ 8us) server_negotiation.cc:407] Connection header received
0807 08:48:07.481396 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:07.481397 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:07.481405 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:07.481461 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:48:07.481546 (+ 85us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:07.482218 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:07.482908 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:07.483172 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:07.483241 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:07.483320 (+ 79us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:48:07.483347 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:48:07.483386 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0807 08:48:07.483420 (+ 34us) server_negotiation.cc:299] Negotiation successful
0807 08:48:07.483437 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 08:48:07.487911 16643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:07.485600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.129:45997 (local address 127.4.159.129:41925)
0807 08:48:07.485634 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:07.485636 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:07.485754 (+ 118us) server_negotiation.cc:407] Connection header received
0807 08:48:07.485814 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:07.485815 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:07.485822 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:07.485862 (+ 40us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:48:07.485963 (+ 101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:07.486402 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:07.487391 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:07.487551 (+ 160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:07.487591 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:07.487745 (+ 154us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:48:07.487764 (+ 19us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:48:07.487791 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0807 08:48:07.487842 (+ 51us) server_negotiation.cc:299] Negotiation successful
0807 08:48:07.487855 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 08:48:07.487990 16741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:07.485531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.129:41925 (local address 127.4.159.129:45997)
0807 08:48:07.485686 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:07.485697 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:07.485762 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:07.485869 (+ 107us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:07.485870 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:07.485871 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 08:48:07.485908 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:07.485910 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:07.486466 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:07.486467 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:07.487318 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:07.487321 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:07.487608 (+ 287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:07.487609 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:07.487662 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:07.487677 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 08:48:07.487795 (+ 118us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 08:48:07.487796 (+ 1us) client_negotiation.cc:769] Sending connection context
0807 08:48:07.487840 (+ 44us) client_negotiation.cc:240] Negotiation successful
0807 08:48:07.487879 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":75,"threads_started":1}
I20260807 08:48:07.489624 16534 raft_consensus.cc:1261] T e18fe512fdfc481bb6784da895df04d1 P 9bcc8a5781ff4eb5bcbe77690fd0c0d4 [term 1 FOLLOWER]: Refusing update from remote peer 4f8ad558036e4bcbbace591712a92519: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:48:07.489637 16317 raft_consensus.cc:1261] T e18fe512fdfc481bb6784da895df04d1 P a6627f8faaf1423fb7504793451b18c3 [term 1 FOLLOWER]: Refusing update from remote peer 4f8ad558036e4bcbbace591712a92519: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:48:07.489810 16701 consensus_queue.cc:1035] T e18fe512fdfc481bb6784da895df04d1 P 4f8ad558036e4bcbbace591712a92519 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6627f8faaf1423fb7504793451b18c3" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 44955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:48:07.489918 16701 consensus_queue.cc:1035] T e18fe512fdfc481bb6784da895df04d1 P 4f8ad558036e4bcbbace591712a92519 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9bcc8a5781ff4eb5bcbe77690fd0c0d4" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 42305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:48:07.502542 15852 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52334:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:48:07.502632 15852 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52334:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:48:07.503269 15852 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4ea7dd9dc02e498b932350ceaf4214cb: Sending DeleteTablet for 3 replicas of tablet e18fe512fdfc481bb6784da895df04d1
I20260807 08:48:07.503516 16504 tablet_service.cc:1514] Processing DeleteTablet for tablet e18fe512fdfc481bb6784da895df04d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:48:07 UTC) from {username='oryx', principal='oryx/127.4.159.190@KRBTEST.COM'} at 127.0.0.1:41800
I20260807 08:48:07.503525 16286 tablet_service.cc:1514] Processing DeleteTablet for tablet e18fe512fdfc481bb6784da895df04d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:48:07 UTC) from {username='oryx', principal='oryx/127.4.159.190@KRBTEST.COM'} at 127.0.0.1:60196
I20260807 08:48:07.503607 16693 tablet_replica.cc:330] T e18fe512fdfc481bb6784da895df04d1 P a6627f8faaf1423fb7504793451b18c3: stopping tablet replica
I20260807 08:48:07.503614 16687 tablet_replica.cc:330] T e18fe512fdfc481bb6784da895df04d1 P 9bcc8a5781ff4eb5bcbe77690fd0c0d4: stopping tablet replica
I20260807 08:48:07.503659 16693 raft_consensus.cc:2229] T e18fe512fdfc481bb6784da895df04d1 P a6627f8faaf1423fb7504793451b18c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:48:07.503688 16687 raft_consensus.cc:2229] T e18fe512fdfc481bb6784da895df04d1 P 9bcc8a5781ff4eb5bcbe77690fd0c0d4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:48:07.503700 16693 raft_consensus.cc:2258] T e18fe512fdfc481bb6784da895df04d1 P a6627f8faaf1423fb7504793451b18c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:48:07.503726 16687 raft_consensus.cc:2258] T e18fe512fdfc481bb6784da895df04d1 P 9bcc8a5781ff4eb5bcbe77690fd0c0d4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:48:07.503923 16068 tablet_service.cc:1514] Processing DeleteTablet for tablet e18fe512fdfc481bb6784da895df04d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:48:07 UTC) from {username='oryx', principal='oryx/127.4.159.190@KRBTEST.COM'} at 127.0.0.1:57072
I20260807 08:48:07.504176 16687 ts_tablet_manager.cc:1905] T e18fe512fdfc481bb6784da895df04d1 P 9bcc8a5781ff4eb5bcbe77690fd0c0d4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:48:07.505626 16687 ts_tablet_manager.cc:1918] T e18fe512fdfc481bb6784da895df04d1 P 9bcc8a5781ff4eb5bcbe77690fd0c0d4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 08:48:07.505686 16687 log.cc:1196] T e18fe512fdfc481bb6784da895df04d1 P 9bcc8a5781ff4eb5bcbe77690fd0c0d4: Deleting WAL directory at /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786092453486433-4734-0/minicluster-data/ts-2/wal/wals/e18fe512fdfc481bb6784da895df04d1
I20260807 08:48:07.505788 16688 tablet_replica.cc:330] T e18fe512fdfc481bb6784da895df04d1 P 4f8ad558036e4bcbbace591712a92519: stopping tablet replica
I20260807 08:48:07.505841 16688 raft_consensus.cc:2229] T e18fe512fdfc481bb6784da895df04d1 P 4f8ad558036e4bcbbace591712a92519 [term 1 LEADER]: Raft consensus shutting down.
I20260807 08:48:07.505889 16687 ts_tablet_manager.cc:1939] T e18fe512fdfc481bb6784da895df04d1 P 9bcc8a5781ff4eb5bcbe77690fd0c0d4: Deleting consensus metadata
I20260807 08:48:07.505919 16688 raft_consensus.cc:2258] T e18fe512fdfc481bb6784da895df04d1 P 4f8ad558036e4bcbbace591712a92519 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:48:07.506255 16688 ts_tablet_manager.cc:1905] T e18fe512fdfc481bb6784da895df04d1 P 4f8ad558036e4bcbbace591712a92519: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:48:07.506405 15829 catalog_manager.cc:4841] TS 9bcc8a5781ff4eb5bcbe77690fd0c0d4 (127.4.159.131:42305): tablet e18fe512fdfc481bb6784da895df04d1 (table test-table [id=d15595b603ac4993a40a24858ff88bd4]) successfully deleted
I20260807 08:48:07.507356 16693 ts_tablet_manager.cc:1905] T e18fe512fdfc481bb6784da895df04d1 P a6627f8faaf1423fb7504793451b18c3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:48:07.507766 16688 ts_tablet_manager.cc:1918] T e18fe512fdfc481bb6784da895df04d1 P 4f8ad558036e4bcbbace591712a92519: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 08:48:07.507833 16688 log.cc:1196] T e18fe512fdfc481bb6784da895df04d1 P 4f8ad558036e4bcbbace591712a92519: Deleting WAL directory at /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786092453486433-4734-0/minicluster-data/ts-0/wal/wals/e18fe512fdfc481bb6784da895df04d1
I20260807 08:48:07.508002 16688 ts_tablet_manager.cc:1939] T e18fe512fdfc481bb6784da895df04d1 P 4f8ad558036e4bcbbace591712a92519: Deleting consensus metadata
I20260807 08:48:07.508680 16693 ts_tablet_manager.cc:1918] T e18fe512fdfc481bb6784da895df04d1 P a6627f8faaf1423fb7504793451b18c3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 08:48:07.508729 16693 log.cc:1196] T e18fe512fdfc481bb6784da895df04d1 P a6627f8faaf1423fb7504793451b18c3: Deleting WAL directory at /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786092453486433-4734-0/minicluster-data/ts-1/wal/wals/e18fe512fdfc481bb6784da895df04d1
I20260807 08:48:07.508930 16693 ts_tablet_manager.cc:1939] T e18fe512fdfc481bb6784da895df04d1 P a6627f8faaf1423fb7504793451b18c3: Deleting consensus metadata
I20260807 08:48:07.509186 15826 catalog_manager.cc:4841] TS 4f8ad558036e4bcbbace591712a92519 (127.4.159.129:41925): tablet e18fe512fdfc481bb6784da895df04d1 (table test-table [id=d15595b603ac4993a40a24858ff88bd4]) successfully deleted
I20260807 08:48:07.509236 15824 catalog_manager.cc:4841] TS a6627f8faaf1423fb7504793451b18c3 (127.4.159.130:44955): tablet e18fe512fdfc481bb6784da895df04d1 (table test-table [id=d15595b603ac4993a40a24858ff88bd4]) successfully deleted
Aug 07 08:48:07 dist-test-slave-6zbq krb5kdc[15775](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092487, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 08:48:07 dist-test-slave-6zbq krb5kdc[15775](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.4.159.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:48:07 dist-test-slave-6zbq krb5kdc[15775](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.4.159.190@KRBTEST.COM, Server not found in Kerberos database
W20260807 08:48:07.529865 15948 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:48:07.524707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52358 (local address 127.4.159.190:45303)
0807 08:48:07.524775 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:07.524778 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:07.524850 (+ 72us) server_negotiation.cc:407] Connection header received
0807 08:48:07.524902 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:07.524903 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:07.524914 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:07.524961 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:07.525472 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:07.526230 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:07.526993 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:07.527283 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:07.527347 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:07.529808 (+ 2461us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52358: BlockingRecv error: recv got EOF from 127.0.0.1:52358 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 07 08:48:07 dist-test-slave-6zbq krb5kdc[15775](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092487, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM
I20260807 08:48:07.540885 15948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:07.530555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52364 (local address 127.4.159.190:45303)
0807 08:48:07.530593 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:07.530595 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:07.534499 (+ 3904us) server_negotiation.cc:407] Connection header received
0807 08:48:07.534528 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:07.534529 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:07.534539 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:07.534603 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:07.535010 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:07.535706 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:07.536362 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:07.536620 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:07.536694 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:07.538625 (+ 1931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:48:07.538627 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:48:07.538627 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:48:07.538633 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:48:07.539854 (+ 1221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:48:07.540231 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:48:07.540231 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:48:07.540232 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:48:07.540276 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:48:07.540452 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:48:07.540453 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:48:07.540453 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:48:07.540539 (+ 86us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:48:07.540601 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 08:48:07.540768 (+ 167us) server_negotiation.cc:299] Negotiation successful
0807 08:48:07.540827 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 08:48:07.542404 15852 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52364:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 08:48:07.542578 15852 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:48:07.544896 16504 tablet_service.cc:1467] Processing CreateTablet for tablet c356984327374074b0e371f8b55f8c25 (DEFAULT_TABLE table=test-table [id=0100dbe8baa14485b2157ac6f4f99818]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:48:07.544948 16286 tablet_service.cc:1467] Processing CreateTablet for tablet c356984327374074b0e371f8b55f8c25 (DEFAULT_TABLE table=test-table [id=0100dbe8baa14485b2157ac6f4f99818]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:48:07.545024 16504 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c356984327374074b0e371f8b55f8c25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:48:07.545094 16286 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c356984327374074b0e371f8b55f8c25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:48:07.546015 16649 tablet_bootstrap.cc:492] T c356984327374074b0e371f8b55f8c25 P 9bcc8a5781ff4eb5bcbe77690fd0c0d4: Bootstrap starting.
I20260807 08:48:07.546379 16649 tablet_bootstrap.cc:654] T c356984327374074b0e371f8b55f8c25 P 9bcc8a5781ff4eb5bcbe77690fd0c0d4: Neither blocks nor log segments found. Creating new log.
I20260807 08:48:07.546433 16068 tablet_service.cc:1467] Processing CreateTablet for tablet c356984327374074b0e371f8b55f8c25 (DEFAULT_TABLE table=test-table [id=0100dbe8baa14485b2157ac6f4f99818]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:48:07.546558 16068 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c356984327374074b0e371f8b55f8c25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:48:07.546881 16649 tablet_bootstrap.cc:492] T c356984327374074b0e371f8b55f8c25 P 9bcc8a5781ff4eb5bcbe77690fd0c0d4: No bootstrap required, opened a new log
I20260807 08:48:07.546928 16649 ts_tablet_manager.cc:1397] T c356984327374074b0e371f8b55f8c25 P 9bcc8a5781ff4eb5bcbe77690fd0c0d4: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260807 08:48:07.547091 16649 raft_consensus.cc:348] T c356984327374074b0e371f8b55f8c25 P 9bcc8a5781ff4eb5bcbe77690fd0c0d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f8ad558036e4bcbbace591712a92519" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 41925 } } peers { permanent_uuid: "a6627f8faaf1423fb7504793451b18c3" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 44955 } } peers { permanent_uuid: "9bcc8a5781ff4eb5bcbe77690fd0c0d4" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 42305 } }
I20260807 08:48:07.547147 16649 raft_consensus.cc:374] T c356984327374074b0e371f8b55f8c25 P 9bcc8a5781ff4eb5bcbe77690fd0c0d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:48:07.547164 16649 raft_consensus.cc:729] T c356984327374074b0e371f8b55f8c25 P 9bcc8a5781ff4eb5bcbe77690fd0c0d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9bcc8a5781ff4eb5bcbe77690fd0c0d4, State: Initialized, Role: FOLLOWER
I20260807 08:48:07.547204 16649 consensus_queue.cc:260] T c356984327374074b0e371f8b55f8c25 P 9bcc8a5781ff4eb5bcbe77690fd0c0d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f8ad558036e4bcbbace591712a92519" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 41925 } } peers { permanent_uuid: "a6627f8faaf1423fb7504793451b18c3" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 44955 } } peers { permanent_uuid: "9bcc8a5781ff4eb5bcbe77690fd0c0d4" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 42305 } }
I20260807 08:48:07.547290 16649 ts_tablet_manager.cc:1428] T c356984327374074b0e371f8b55f8c25 P 9bcc8a5781ff4eb5bcbe77690fd0c0d4: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 08:48:07.547848 16653 tablet_bootstrap.cc:492] T c356984327374074b0e371f8b55f8c25 P 4f8ad558036e4bcbbace591712a92519: Bootstrap starting.
I20260807 08:48:07.548198 16653 tablet_bootstrap.cc:654] T c356984327374074b0e371f8b55f8c25 P 4f8ad558036e4bcbbace591712a92519: Neither blocks nor log segments found. Creating new log.
I20260807 08:48:07.548302 16648 tablet_bootstrap.cc:492] T c356984327374074b0e371f8b55f8c25 P a6627f8faaf1423fb7504793451b18c3: Bootstrap starting.
I20260807 08:48:07.548679 16648 tablet_bootstrap.cc:654] T c356984327374074b0e371f8b55f8c25 P a6627f8faaf1423fb7504793451b18c3: Neither blocks nor log segments found. Creating new log.
I20260807 08:48:07.548753 16653 tablet_bootstrap.cc:492] T c356984327374074b0e371f8b55f8c25 P 4f8ad558036e4bcbbace591712a92519: No bootstrap required, opened a new log
I20260807 08:48:07.548801 16653 ts_tablet_manager.cc:1397] T c356984327374074b0e371f8b55f8c25 P 4f8ad558036e4bcbbace591712a92519: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260807 08:48:07.548946 16653 raft_consensus.cc:348] T c356984327374074b0e371f8b55f8c25 P 4f8ad558036e4bcbbace591712a92519 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f8ad558036e4bcbbace591712a92519" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 41925 } } peers { permanent_uuid: "a6627f8faaf1423fb7504793451b18c3" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 44955 } } peers { permanent_uuid: "9bcc8a5781ff4eb5bcbe77690fd0c0d4" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 42305 } }
I20260807 08:48:07.549029 16653 raft_consensus.cc:374] T c356984327374074b0e371f8b55f8c25 P 4f8ad558036e4bcbbace591712a92519 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:48:07.549054 16653 raft_consensus.cc:729] T c356984327374074b0e371f8b55f8c25 P 4f8ad558036e4bcbbace591712a92519 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f8ad558036e4bcbbace591712a92519, State: Initialized, Role: FOLLOWER
I20260807 08:48:07.549093 16653 consensus_queue.cc:260] T c356984327374074b0e371f8b55f8c25 P 4f8ad558036e4bcbbace591712a92519 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f8ad558036e4bcbbace591712a92519" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 41925 } } peers { permanent_uuid: "a6627f8faaf1423fb7504793451b18c3" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 44955 } } peers { permanent_uuid: "9bcc8a5781ff4eb5bcbe77690fd0c0d4" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 42305 } }
I20260807 08:48:07.549175 16653 ts_tablet_manager.cc:1428] T c356984327374074b0e371f8b55f8c25 P 4f8ad558036e4bcbbace591712a92519: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 08:48:07.549520 16648 tablet_bootstrap.cc:492] T c356984327374074b0e371f8b55f8c25 P a6627f8faaf1423fb7504793451b18c3: No bootstrap required, opened a new log
I20260807 08:48:07.549579 16648 ts_tablet_manager.cc:1397] T c356984327374074b0e371f8b55f8c25 P a6627f8faaf1423fb7504793451b18c3: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 08:48:07.549732 16648 raft_consensus.cc:348] T c356984327374074b0e371f8b55f8c25 P a6627f8faaf1423fb7504793451b18c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f8ad558036e4bcbbace591712a92519" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 41925 } } peers { permanent_uuid: "a6627f8faaf1423fb7504793451b18c3" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 44955 } } peers { permanent_uuid: "9bcc8a5781ff4eb5bcbe77690fd0c0d4" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 42305 } }
I20260807 08:48:07.549809 16648 raft_consensus.cc:374] T c356984327374074b0e371f8b55f8c25 P a6627f8faaf1423fb7504793451b18c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:48:07.549831 16648 raft_consensus.cc:729] T c356984327374074b0e371f8b55f8c25 P a6627f8faaf1423fb7504793451b18c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6627f8faaf1423fb7504793451b18c3, State: Initialized, Role: FOLLOWER
I20260807 08:48:07.549872 16648 consensus_queue.cc:260] T c356984327374074b0e371f8b55f8c25 P a6627f8faaf1423fb7504793451b18c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f8ad558036e4bcbbace591712a92519" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 41925 } } peers { permanent_uuid: "a6627f8faaf1423fb7504793451b18c3" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 44955 } } peers { permanent_uuid: "9bcc8a5781ff4eb5bcbe77690fd0c0d4" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 42305 } }
I20260807 08:48:07.549968 16648 ts_tablet_manager.cc:1428] T c356984327374074b0e371f8b55f8c25 P a6627f8faaf1423fb7504793451b18c3: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 08:48:07.565104 16654 raft_consensus.cc:482] T c356984327374074b0e371f8b55f8c25 P a6627f8faaf1423fb7504793451b18c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:48:07.565186 16654 raft_consensus.cc:504] T c356984327374074b0e371f8b55f8c25 P a6627f8faaf1423fb7504793451b18c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f8ad558036e4bcbbace591712a92519" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 41925 } } peers { permanent_uuid: "a6627f8faaf1423fb7504793451b18c3" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 44955 } } peers { permanent_uuid: "9bcc8a5781ff4eb5bcbe77690fd0c0d4" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 42305 } }
I20260807 08:48:07.565518 16654 leader_election.cc:288] T c356984327374074b0e371f8b55f8c25 P a6627f8faaf1423fb7504793451b18c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f8ad558036e4bcbbace591712a92519 (127.4.159.129:41925), 9bcc8a5781ff4eb5bcbe77690fd0c0d4 (127.4.159.131:42305)
I20260807 08:48:07.568938 16643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:07.565781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.130:48175 (local address 127.4.159.129:41925)
0807 08:48:07.565840 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:07.565842 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:07.565925 (+ 83us) server_negotiation.cc:407] Connection header received
0807 08:48:07.566005 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:07.566007 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:07.566014 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:07.566038 (+ 24us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:48:07.566242 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:07.566929 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:07.568064 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:07.568578 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:07.568648 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:07.568657 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 08:48:07.568845 (+ 188us) server_negotiation.cc:299] Negotiation successful
0807 08:48:07.568878 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 08:48:07.568939 16770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:07.565646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.129:41925 (local address 127.4.159.130:48175)
0807 08:48:07.565867 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:07.565876 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:07.565937 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:07.566089 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:07.566089 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:07.566091 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:48:07.566178 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:07.566180 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:07.566987 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:07.566988 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:07.567983 (+ 995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:07.567985 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:07.568685 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:07.568686 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:07.568767 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:07.568771 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 08:48:07.568826 (+ 55us) client_negotiation.cc:240] Negotiation successful
0807 08:48:07.568864 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":38,"threads_started":1}
I20260807 08:48:07.569000 16646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:07.565714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.130:37057 (local address 127.4.159.131:42305)
0807 08:48:07.565750 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:07.565751 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:07.565812 (+ 61us) server_negotiation.cc:407] Connection header received
0807 08:48:07.565906 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:07.565907 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:07.565918 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:07.565964 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:48:07.566092 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:07.566836 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:07.568162 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:07.568687 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:07.568738 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:07.568746 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0807 08:48:07.568910 (+ 164us) server_negotiation.cc:299] Negotiation successful
0807 08:48:07.568950 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 08:48:07.569200 16767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:07.565632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.131:42305 (local address 127.4.159.130:37057)
0807 08:48:07.565761 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:07.565770 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:07.565823 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:07.565968 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:07.565968 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:07.565971 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:48:07.566032 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:07.566034 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:07.566916 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:07.566917 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:07.568069 (+ 1152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:07.568072 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:07.568754 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:07.568755 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:07.568833 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:07.568837 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 08:48:07.569082 (+ 245us) client_negotiation.cc:240] Negotiation successful
0807 08:48:07.569126 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":55,"threads_started":1}
I20260807 08:48:07.569197 16097 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c356984327374074b0e371f8b55f8c25" candidate_uuid: "a6627f8faaf1423fb7504793451b18c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f8ad558036e4bcbbace591712a92519" is_pre_election: true
I20260807 08:48:07.569267 16097 raft_consensus.cc:2454] T c356984327374074b0e371f8b55f8c25 P 4f8ad558036e4bcbbace591712a92519 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a6627f8faaf1423fb7504793451b18c3 in term 0.
I20260807 08:48:07.569401 16214 leader_election.cc:302] T c356984327374074b0e371f8b55f8c25 P a6627f8faaf1423fb7504793451b18c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f8ad558036e4bcbbace591712a92519, a6627f8faaf1423fb7504793451b18c3; no voters:
I20260807 08:48:07.569442 16534 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c356984327374074b0e371f8b55f8c25" candidate_uuid: "a6627f8faaf1423fb7504793451b18c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9bcc8a5781ff4eb5bcbe77690fd0c0d4" is_pre_election: true
I20260807 08:48:07.569517 16534 raft_consensus.cc:2454] T c356984327374074b0e371f8b55f8c25 P 9bcc8a5781ff4eb5bcbe77690fd0c0d4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a6627f8faaf1423fb7504793451b18c3 in term 0.
I20260807 08:48:07.569506 16654 raft_consensus.cc:2781] T c356984327374074b0e371f8b55f8c25 P a6627f8faaf1423fb7504793451b18c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:48:07.569551 16654 raft_consensus.cc:482] T c356984327374074b0e371f8b55f8c25 P a6627f8faaf1423fb7504793451b18c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:48:07.569578 16654 raft_consensus.cc:3037] T c356984327374074b0e371f8b55f8c25 P a6627f8faaf1423fb7504793451b18c3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:48:07.570331 16654 raft_consensus.cc:504] T c356984327374074b0e371f8b55f8c25 P a6627f8faaf1423fb7504793451b18c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f8ad558036e4bcbbace591712a92519" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 41925 } } peers { permanent_uuid: "a6627f8faaf1423fb7504793451b18c3" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 44955 } } peers { permanent_uuid: "9bcc8a5781ff4eb5bcbe77690fd0c0d4" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 42305 } }
I20260807 08:48:07.570488 16654 leader_election.cc:288] T c356984327374074b0e371f8b55f8c25 P a6627f8faaf1423fb7504793451b18c3 [CANDIDATE]: Term 1 election: Requested vote from peers 4f8ad558036e4bcbbace591712a92519 (127.4.159.129:41925), 9bcc8a5781ff4eb5bcbe77690fd0c0d4 (127.4.159.131:42305)
I20260807 08:48:07.570618 16097 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c356984327374074b0e371f8b55f8c25" candidate_uuid: "a6627f8faaf1423fb7504793451b18c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f8ad558036e4bcbbace591712a92519"
I20260807 08:48:07.570694 16097 raft_consensus.cc:3037] T c356984327374074b0e371f8b55f8c25 P 4f8ad558036e4bcbbace591712a92519 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:48:07.570664 16534 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c356984327374074b0e371f8b55f8c25" candidate_uuid: "a6627f8faaf1423fb7504793451b18c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9bcc8a5781ff4eb5bcbe77690fd0c0d4"
I20260807 08:48:07.570730 16534 raft_consensus.cc:3037] T c356984327374074b0e371f8b55f8c25 P 9bcc8a5781ff4eb5bcbe77690fd0c0d4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:48:07.571192 16097 raft_consensus.cc:2454] T c356984327374074b0e371f8b55f8c25 P 4f8ad558036e4bcbbace591712a92519 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6627f8faaf1423fb7504793451b18c3 in term 1.
I20260807 08:48:07.571354 16214 leader_election.cc:302] T c356984327374074b0e371f8b55f8c25 P a6627f8faaf1423fb7504793451b18c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f8ad558036e4bcbbace591712a92519, a6627f8faaf1423fb7504793451b18c3; no voters:
I20260807 08:48:07.571439 16534 raft_consensus.cc:2454] T c356984327374074b0e371f8b55f8c25 P 9bcc8a5781ff4eb5bcbe77690fd0c0d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6627f8faaf1423fb7504793451b18c3 in term 1.
I20260807 08:48:07.571540 16654 raft_consensus.cc:2781] T c356984327374074b0e371f8b55f8c25 P a6627f8faaf1423fb7504793451b18c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:48:07.571638 16654 raft_consensus.cc:686] T c356984327374074b0e371f8b55f8c25 P a6627f8faaf1423fb7504793451b18c3 [term 1 LEADER]: Becoming Leader. State: Replica: a6627f8faaf1423fb7504793451b18c3, State: Running, Role: LEADER
I20260807 08:48:07.571698 16654 consensus_queue.cc:237] T c356984327374074b0e371f8b55f8c25 P a6627f8faaf1423fb7504793451b18c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f8ad558036e4bcbbace591712a92519" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 41925 } } peers { permanent_uuid: "a6627f8faaf1423fb7504793451b18c3" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 44955 } } peers { permanent_uuid: "9bcc8a5781ff4eb5bcbe77690fd0c0d4" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 42305 } }
I20260807 08:48:07.572346 15852 catalog_manager.cc:5495] T c356984327374074b0e371f8b55f8c25 P a6627f8faaf1423fb7504793451b18c3 reported cstate change: term changed from 0 to 1, leader changed from <none> to a6627f8faaf1423fb7504793451b18c3 (127.4.159.130). New cstate: current_term: 1 leader_uuid: "a6627f8faaf1423fb7504793451b18c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f8ad558036e4bcbbace591712a92519" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 41925 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a6627f8faaf1423fb7504793451b18c3" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 44955 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9bcc8a5781ff4eb5bcbe77690fd0c0d4" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 42305 } health_report { overall_health: UNKNOWN } } }
I20260807 08:48:07.580497 16645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:07.578172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60198 (local address 127.4.159.130:44955)
0807 08:48:07.578210 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:07.578212 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:07.578220 (+ 8us) server_negotiation.cc:407] Connection header received
0807 08:48:07.578270 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:07.578271 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:07.578290 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:07.578381 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:48:07.578464 (+ 83us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:07.579149 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:07.579785 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:07.580074 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:07.580143 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:07.580227 (+ 84us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:48:07.580337 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:48:07.580401 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 08:48:07.580416 (+ 15us) server_negotiation.cc:299] Negotiation successful
0807 08:48:07.580434 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 08:48:07.581280 16097 raft_consensus.cc:1261] T c356984327374074b0e371f8b55f8c25 P 4f8ad558036e4bcbbace591712a92519 [term 1 FOLLOWER]: Refusing update from remote peer a6627f8faaf1423fb7504793451b18c3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:48:07.581408 16534 raft_consensus.cc:1261] T c356984327374074b0e371f8b55f8c25 P 9bcc8a5781ff4eb5bcbe77690fd0c0d4 [term 1 FOLLOWER]: Refusing update from remote peer a6627f8faaf1423fb7504793451b18c3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:48:07.581573 16771 consensus_queue.cc:1035] T c356984327374074b0e371f8b55f8c25 P a6627f8faaf1423fb7504793451b18c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f8ad558036e4bcbbace591712a92519" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 41925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:48:07.581689 16654 consensus_queue.cc:1035] T c356984327374074b0e371f8b55f8c25 P a6627f8faaf1423fb7504793451b18c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9bcc8a5781ff4eb5bcbe77690fd0c0d4" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 42305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:48:07.584923 15846 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52364:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:48:07.584995 15846 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52364:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:48:07.585562 15846 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4ea7dd9dc02e498b932350ceaf4214cb: Sending DeleteTablet for 3 replicas of tablet c356984327374074b0e371f8b55f8c25
I20260807 08:48:07.585865 16286 tablet_service.cc:1514] Processing DeleteTablet for tablet c356984327374074b0e371f8b55f8c25 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:48:07 UTC) from {username='oryx', principal='oryx/127.4.159.190@KRBTEST.COM'} at 127.0.0.1:60196
I20260807 08:48:07.585879 16504 tablet_service.cc:1514] Processing DeleteTablet for tablet c356984327374074b0e371f8b55f8c25 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:48:07 UTC) from {username='oryx', principal='oryx/127.4.159.190@KRBTEST.COM'} at 127.0.0.1:41800
I20260807 08:48:07.585942 16693 tablet_replica.cc:330] T c356984327374074b0e371f8b55f8c25 P a6627f8faaf1423fb7504793451b18c3: stopping tablet replica
I20260807 08:48:07.585963 16687 tablet_replica.cc:330] T c356984327374074b0e371f8b55f8c25 P 9bcc8a5781ff4eb5bcbe77690fd0c0d4: stopping tablet replica
I20260807 08:48:07.585983 16693 raft_consensus.cc:2229] T c356984327374074b0e371f8b55f8c25 P a6627f8faaf1423fb7504793451b18c3 [term 1 LEADER]: Raft consensus shutting down.
I20260807 08:48:07.586006 16687 raft_consensus.cc:2229] T c356984327374074b0e371f8b55f8c25 P 9bcc8a5781ff4eb5bcbe77690fd0c0d4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:48:07.586043 16687 raft_consensus.cc:2258] T c356984327374074b0e371f8b55f8c25 P 9bcc8a5781ff4eb5bcbe77690fd0c0d4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:48:07.586053 16693 raft_consensus.cc:2258] T c356984327374074b0e371f8b55f8c25 P a6627f8faaf1423fb7504793451b18c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:48:07.586308 16687 ts_tablet_manager.cc:1905] T c356984327374074b0e371f8b55f8c25 P 9bcc8a5781ff4eb5bcbe77690fd0c0d4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:48:07.586477 16693 ts_tablet_manager.cc:1905] T c356984327374074b0e371f8b55f8c25 P a6627f8faaf1423fb7504793451b18c3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:48:07.587653 16687 ts_tablet_manager.cc:1918] T c356984327374074b0e371f8b55f8c25 P 9bcc8a5781ff4eb5bcbe77690fd0c0d4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:48:07.587703 16687 log.cc:1196] T c356984327374074b0e371f8b55f8c25 P 9bcc8a5781ff4eb5bcbe77690fd0c0d4: Deleting WAL directory at /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786092453486433-4734-0/minicluster-data/ts-2/wal/wals/c356984327374074b0e371f8b55f8c25
I20260807 08:48:07.587749 16693 ts_tablet_manager.cc:1918] T c356984327374074b0e371f8b55f8c25 P a6627f8faaf1423fb7504793451b18c3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:48:07.587800 16693 log.cc:1196] T c356984327374074b0e371f8b55f8c25 P a6627f8faaf1423fb7504793451b18c3: Deleting WAL directory at /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786092453486433-4734-0/minicluster-data/ts-1/wal/wals/c356984327374074b0e371f8b55f8c25
I20260807 08:48:07.587901 16687 ts_tablet_manager.cc:1939] T c356984327374074b0e371f8b55f8c25 P 9bcc8a5781ff4eb5bcbe77690fd0c0d4: Deleting consensus metadata
I20260807 08:48:07.587990 16693 ts_tablet_manager.cc:1939] T c356984327374074b0e371f8b55f8c25 P a6627f8faaf1423fb7504793451b18c3: Deleting consensus metadata
I20260807 08:48:07.588212 15829 catalog_manager.cc:4841] TS 9bcc8a5781ff4eb5bcbe77690fd0c0d4 (127.4.159.131:42305): tablet c356984327374074b0e371f8b55f8c25 (table test-table [id=0100dbe8baa14485b2157ac6f4f99818]) successfully deleted
I20260807 08:48:07.588272 15824 catalog_manager.cc:4841] TS a6627f8faaf1423fb7504793451b18c3 (127.4.159.130:44955): tablet c356984327374074b0e371f8b55f8c25 (table test-table [id=0100dbe8baa14485b2157ac6f4f99818]) successfully deleted
I20260807 08:48:07.589315 16068 tablet_service.cc:1514] Processing DeleteTablet for tablet c356984327374074b0e371f8b55f8c25 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:48:07 UTC) from {username='oryx', principal='oryx/127.4.159.190@KRBTEST.COM'} at 127.0.0.1:57072
I20260807 08:48:07.589399 16688 tablet_replica.cc:330] T c356984327374074b0e371f8b55f8c25 P 4f8ad558036e4bcbbace591712a92519: stopping tablet replica
I20260807 08:48:07.589438 16688 raft_consensus.cc:2229] T c356984327374074b0e371f8b55f8c25 P 4f8ad558036e4bcbbace591712a92519 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:48:07.589473 16688 raft_consensus.cc:2258] T c356984327374074b0e371f8b55f8c25 P 4f8ad558036e4bcbbace591712a92519 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:48:07.589766 16688 ts_tablet_manager.cc:1905] T c356984327374074b0e371f8b55f8c25 P 4f8ad558036e4bcbbace591712a92519: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:48:07.589748 15846 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52364:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 08:48:07.589890 15846 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:48:07.591022 16504 tablet_service.cc:1467] Processing CreateTablet for tablet 2584a398d3f6437fbf76c21addd6ce1d (DEFAULT_TABLE table=test-table [id=7e54383015db4909b3a5bad4e4b1d372]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:48:07.591104 16688 ts_tablet_manager.cc:1918] T c356984327374074b0e371f8b55f8c25 P 4f8ad558036e4bcbbace591712a92519: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:48:07.591141 16504 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2584a398d3f6437fbf76c21addd6ce1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:48:07.591156 16688 log.cc:1196] T c356984327374074b0e371f8b55f8c25 P 4f8ad558036e4bcbbace591712a92519: Deleting WAL directory at /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786092453486433-4734-0/minicluster-data/ts-0/wal/wals/c356984327374074b0e371f8b55f8c25
I20260807 08:48:07.591358 16688 ts_tablet_manager.cc:1939] T c356984327374074b0e371f8b55f8c25 P 4f8ad558036e4bcbbace591712a92519: Deleting consensus metadata
I20260807 08:48:07.592196 16649 tablet_bootstrap.cc:492] T 2584a398d3f6437fbf76c21addd6ce1d P 9bcc8a5781ff4eb5bcbe77690fd0c0d4: Bootstrap starting.
I20260807 08:48:07.592474 16649 tablet_bootstrap.cc:654] T 2584a398d3f6437fbf76c21addd6ce1d P 9bcc8a5781ff4eb5bcbe77690fd0c0d4: Neither blocks nor log segments found. Creating new log.
I20260807 08:48:07.592903 16286 tablet_service.cc:1467] Processing CreateTablet for tablet 2584a398d3f6437fbf76c21addd6ce1d (DEFAULT_TABLE table=test-table [id=7e54383015db4909b3a5bad4e4b1d372]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:48:07.592995 16286 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2584a398d3f6437fbf76c21addd6ce1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:48:07.594009 16648 tablet_bootstrap.cc:492] T 2584a398d3f6437fbf76c21addd6ce1d P a6627f8faaf1423fb7504793451b18c3: Bootstrap starting.
I20260807 08:48:07.594419 16648 tablet_bootstrap.cc:654] T 2584a398d3f6437fbf76c21addd6ce1d P a6627f8faaf1423fb7504793451b18c3: Neither blocks nor log segments found. Creating new log.
I20260807 08:48:07.594656 15826 catalog_manager.cc:4841] TS 4f8ad558036e4bcbbace591712a92519 (127.4.159.129:41925): tablet c356984327374074b0e371f8b55f8c25 (table test-table [id=0100dbe8baa14485b2157ac6f4f99818]) successfully deleted
I20260807 08:48:07.594807 16068 tablet_service.cc:1467] Processing CreateTablet for tablet 2584a398d3f6437fbf76c21addd6ce1d (DEFAULT_TABLE table=test-table [id=7e54383015db4909b3a5bad4e4b1d372]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:48:07.594893 16068 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2584a398d3f6437fbf76c21addd6ce1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:48:07.595355 16648 tablet_bootstrap.cc:492] T 2584a398d3f6437fbf76c21addd6ce1d P a6627f8faaf1423fb7504793451b18c3: No bootstrap required, opened a new log
I20260807 08:48:07.595394 16648 ts_tablet_manager.cc:1397] T 2584a398d3f6437fbf76c21addd6ce1d P a6627f8faaf1423fb7504793451b18c3: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260807 08:48:07.595515 16648 raft_consensus.cc:348] T 2584a398d3f6437fbf76c21addd6ce1d P a6627f8faaf1423fb7504793451b18c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f8ad558036e4bcbbace591712a92519" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 41925 } } peers { permanent_uuid: "a6627f8faaf1423fb7504793451b18c3" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 44955 } } peers { permanent_uuid: "9bcc8a5781ff4eb5bcbe77690fd0c0d4" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 42305 } }
I20260807 08:48:07.595567 16648 raft_consensus.cc:374] T 2584a398d3f6437fbf76c21addd6ce1d P a6627f8faaf1423fb7504793451b18c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:48:07.595585 16648 raft_consensus.cc:729] T 2584a398d3f6437fbf76c21addd6ce1d P a6627f8faaf1423fb7504793451b18c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6627f8faaf1423fb7504793451b18c3, State: Initialized, Role: FOLLOWER
I20260807 08:48:07.595621 16648 consensus_queue.cc:260] T 2584a398d3f6437fbf76c21addd6ce1d P a6627f8faaf1423fb7504793451b18c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f8ad558036e4bcbbace591712a92519" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 41925 } } peers { permanent_uuid: "a6627f8faaf1423fb7504793451b18c3" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 44955 } } peers { permanent_uuid: "9bcc8a5781ff4eb5bcbe77690fd0c0d4" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 42305 } }
I20260807 08:48:07.595696 16648 ts_tablet_manager.cc:1428] T 2584a398d3f6437fbf76c21addd6ce1d P a6627f8faaf1423fb7504793451b18c3: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 08:48:07.595913 16653 tablet_bootstrap.cc:492] T 2584a398d3f6437fbf76c21addd6ce1d P 4f8ad558036e4bcbbace591712a92519: Bootstrap starting.
I20260807 08:48:07.596200 16653 tablet_bootstrap.cc:654] T 2584a398d3f6437fbf76c21addd6ce1d P 4f8ad558036e4bcbbace591712a92519: Neither blocks nor log segments found. Creating new log.
I20260807 08:48:07.596688 16649 tablet_bootstrap.cc:492] T 2584a398d3f6437fbf76c21addd6ce1d P 9bcc8a5781ff4eb5bcbe77690fd0c0d4: No bootstrap required, opened a new log
I20260807 08:48:07.596740 16649 ts_tablet_manager.cc:1397] T 2584a398d3f6437fbf76c21addd6ce1d P 9bcc8a5781ff4eb5bcbe77690fd0c0d4: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.000s
I20260807 08:48:07.596837 16649 raft_consensus.cc:348] T 2584a398d3f6437fbf76c21addd6ce1d P 9bcc8a5781ff4eb5bcbe77690fd0c0d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f8ad558036e4bcbbace591712a92519" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 41925 } } peers { permanent_uuid: "a6627f8faaf1423fb7504793451b18c3" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 44955 } } peers { permanent_uuid: "9bcc8a5781ff4eb5bcbe77690fd0c0d4" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 42305 } }
I20260807 08:48:07.596896 16649 raft_consensus.cc:374] T 2584a398d3f6437fbf76c21addd6ce1d P 9bcc8a5781ff4eb5bcbe77690fd0c0d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:48:07.596921 16649 raft_consensus.cc:729] T 2584a398d3f6437fbf76c21addd6ce1d P 9bcc8a5781ff4eb5bcbe77690fd0c0d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9bcc8a5781ff4eb5bcbe77690fd0c0d4, State: Initialized, Role: FOLLOWER
I20260807 08:48:07.596966 16649 consensus_queue.cc:260] T 2584a398d3f6437fbf76c21addd6ce1d P 9bcc8a5781ff4eb5bcbe77690fd0c0d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f8ad558036e4bcbbace591712a92519" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 41925 } } peers { permanent_uuid: "a6627f8faaf1423fb7504793451b18c3" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 44955 } } peers { permanent_uuid: "9bcc8a5781ff4eb5bcbe77690fd0c0d4" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 42305 } }
I20260807 08:48:07.597045 16649 ts_tablet_manager.cc:1428] T 2584a398d3f6437fbf76c21addd6ce1d P 9bcc8a5781ff4eb5bcbe77690fd0c0d4: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 08:48:07.597862 16653 tablet_bootstrap.cc:492] T 2584a398d3f6437fbf76c21addd6ce1d P 4f8ad558036e4bcbbace591712a92519: No bootstrap required, opened a new log
I20260807 08:48:07.597913 16653 ts_tablet_manager.cc:1397] T 2584a398d3f6437fbf76c21addd6ce1d P 4f8ad558036e4bcbbace591712a92519: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20260807 08:48:07.597998 16653 raft_consensus.cc:348] T 2584a398d3f6437fbf76c21addd6ce1d P 4f8ad558036e4bcbbace591712a92519 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f8ad558036e4bcbbace591712a92519" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 41925 } } peers { permanent_uuid: "a6627f8faaf1423fb7504793451b18c3" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 44955 } } peers { permanent_uuid: "9bcc8a5781ff4eb5bcbe77690fd0c0d4" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 42305 } }
I20260807 08:48:07.598060 16653 raft_consensus.cc:374] T 2584a398d3f6437fbf76c21addd6ce1d P 4f8ad558036e4bcbbace591712a92519 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:48:07.598079 16653 raft_consensus.cc:729] T 2584a398d3f6437fbf76c21addd6ce1d P 4f8ad558036e4bcbbace591712a92519 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f8ad558036e4bcbbace591712a92519, State: Initialized, Role: FOLLOWER
I20260807 08:48:07.598122 16653 consensus_queue.cc:260] T 2584a398d3f6437fbf76c21addd6ce1d P 4f8ad558036e4bcbbace591712a92519 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f8ad558036e4bcbbace591712a92519" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 41925 } } peers { permanent_uuid: "a6627f8faaf1423fb7504793451b18c3" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 44955 } } peers { permanent_uuid: "9bcc8a5781ff4eb5bcbe77690fd0c0d4" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 42305 } }
I20260807 08:48:07.598213 16653 ts_tablet_manager.cc:1428] T 2584a398d3f6437fbf76c21addd6ce1d P 4f8ad558036e4bcbbace591712a92519: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 08:48:07.905923 16679 raft_consensus.cc:482] T 2584a398d3f6437fbf76c21addd6ce1d P 9bcc8a5781ff4eb5bcbe77690fd0c0d4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:48:07.906013 16679 raft_consensus.cc:504] T 2584a398d3f6437fbf76c21addd6ce1d P 9bcc8a5781ff4eb5bcbe77690fd0c0d4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f8ad558036e4bcbbace591712a92519" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 41925 } } peers { permanent_uuid: "a6627f8faaf1423fb7504793451b18c3" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 44955 } } peers { permanent_uuid: "9bcc8a5781ff4eb5bcbe77690fd0c0d4" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 42305 } }
I20260807 08:48:07.906165 16679 leader_election.cc:288] T 2584a398d3f6437fbf76c21addd6ce1d P 9bcc8a5781ff4eb5bcbe77690fd0c0d4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f8ad558036e4bcbbace591712a92519 (127.4.159.129:41925), a6627f8faaf1423fb7504793451b18c3 (127.4.159.130:44955)
I20260807 08:48:07.906419 16317 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2584a398d3f6437fbf76c21addd6ce1d" candidate_uuid: "9bcc8a5781ff4eb5bcbe77690fd0c0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6627f8faaf1423fb7504793451b18c3" is_pre_election: true
I20260807 08:48:07.906421 16097 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2584a398d3f6437fbf76c21addd6ce1d" candidate_uuid: "9bcc8a5781ff4eb5bcbe77690fd0c0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f8ad558036e4bcbbace591712a92519" is_pre_election: true
I20260807 08:48:07.906531 16097 raft_consensus.cc:2454] T 2584a398d3f6437fbf76c21addd6ce1d P 4f8ad558036e4bcbbace591712a92519 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9bcc8a5781ff4eb5bcbe77690fd0c0d4 in term 0.
I20260807 08:48:07.906531 16317 raft_consensus.cc:2454] T 2584a398d3f6437fbf76c21addd6ce1d P a6627f8faaf1423fb7504793451b18c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9bcc8a5781ff4eb5bcbe77690fd0c0d4 in term 0.
I20260807 08:48:07.906666 16425 leader_election.cc:302] T 2584a398d3f6437fbf76c21addd6ce1d P 9bcc8a5781ff4eb5bcbe77690fd0c0d4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9bcc8a5781ff4eb5bcbe77690fd0c0d4, a6627f8faaf1423fb7504793451b18c3; no voters:
I20260807 08:48:07.906781 16679 raft_consensus.cc:2781] T 2584a398d3f6437fbf76c21addd6ce1d P 9bcc8a5781ff4eb5bcbe77690fd0c0d4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:48:07.906822 16679 raft_consensus.cc:482] T 2584a398d3f6437fbf76c21addd6ce1d P 9bcc8a5781ff4eb5bcbe77690fd0c0d4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:48:07.906839 16679 raft_consensus.cc:3037] T 2584a398d3f6437fbf76c21addd6ce1d P 9bcc8a5781ff4eb5bcbe77690fd0c0d4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:48:07.907469 16679 raft_consensus.cc:504] T 2584a398d3f6437fbf76c21addd6ce1d P 9bcc8a5781ff4eb5bcbe77690fd0c0d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f8ad558036e4bcbbace591712a92519" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 41925 } } peers { permanent_uuid: "a6627f8faaf1423fb7504793451b18c3" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 44955 } } peers { permanent_uuid: "9bcc8a5781ff4eb5bcbe77690fd0c0d4" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 42305 } }
I20260807 08:48:07.907589 16679 leader_election.cc:288] T 2584a398d3f6437fbf76c21addd6ce1d P 9bcc8a5781ff4eb5bcbe77690fd0c0d4 [CANDIDATE]: Term 1 election: Requested vote from peers 4f8ad558036e4bcbbace591712a92519 (127.4.159.129:41925), a6627f8faaf1423fb7504793451b18c3 (127.4.159.130:44955)
I20260807 08:48:07.907759 16097 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2584a398d3f6437fbf76c21addd6ce1d" candidate_uuid: "9bcc8a5781ff4eb5bcbe77690fd0c0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f8ad558036e4bcbbace591712a92519"
I20260807 08:48:07.907835 16097 raft_consensus.cc:3037] T 2584a398d3f6437fbf76c21addd6ce1d P 4f8ad558036e4bcbbace591712a92519 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:48:07.907765 16317 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2584a398d3f6437fbf76c21addd6ce1d" candidate_uuid: "9bcc8a5781ff4eb5bcbe77690fd0c0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6627f8faaf1423fb7504793451b18c3"
I20260807 08:48:07.907869 16317 raft_consensus.cc:3037] T 2584a398d3f6437fbf76c21addd6ce1d P a6627f8faaf1423fb7504793451b18c3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:48:07.908442 16097 raft_consensus.cc:2454] T 2584a398d3f6437fbf76c21addd6ce1d P 4f8ad558036e4bcbbace591712a92519 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9bcc8a5781ff4eb5bcbe77690fd0c0d4 in term 1.
I20260807 08:48:07.908545 16317 raft_consensus.cc:2454] T 2584a398d3f6437fbf76c21addd6ce1d P a6627f8faaf1423fb7504793451b18c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9bcc8a5781ff4eb5bcbe77690fd0c0d4 in term 1.
I20260807 08:48:07.908610 16426 leader_election.cc:302] T 2584a398d3f6437fbf76c21addd6ce1d P 9bcc8a5781ff4eb5bcbe77690fd0c0d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f8ad558036e4bcbbace591712a92519, 9bcc8a5781ff4eb5bcbe77690fd0c0d4; no voters:
I20260807 08:48:07.908691 16679 raft_consensus.cc:2781] T 2584a398d3f6437fbf76c21addd6ce1d P 9bcc8a5781ff4eb5bcbe77690fd0c0d4 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:48:07.908795 16679 raft_consensus.cc:686] T 2584a398d3f6437fbf76c21addd6ce1d P 9bcc8a5781ff4eb5bcbe77690fd0c0d4 [term 1 LEADER]: Becoming Leader. State: Replica: 9bcc8a5781ff4eb5bcbe77690fd0c0d4, State: Running, Role: LEADER
I20260807 08:48:07.908849 16679 consensus_queue.cc:237] T 2584a398d3f6437fbf76c21addd6ce1d P 9bcc8a5781ff4eb5bcbe77690fd0c0d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f8ad558036e4bcbbace591712a92519" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 41925 } } peers { permanent_uuid: "a6627f8faaf1423fb7504793451b18c3" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 44955 } } peers { permanent_uuid: "9bcc8a5781ff4eb5bcbe77690fd0c0d4" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 42305 } }
I20260807 08:48:07.909406 15846 catalog_manager.cc:5495] T 2584a398d3f6437fbf76c21addd6ce1d P 9bcc8a5781ff4eb5bcbe77690fd0c0d4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9bcc8a5781ff4eb5bcbe77690fd0c0d4 (127.4.159.131). New cstate: current_term: 1 leader_uuid: "9bcc8a5781ff4eb5bcbe77690fd0c0d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f8ad558036e4bcbbace591712a92519" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 41925 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a6627f8faaf1423fb7504793451b18c3" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 44955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9bcc8a5781ff4eb5bcbe77690fd0c0d4" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 42305 } health_report { overall_health: HEALTHY } } }
I20260807 08:48:07.953683 16646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:07.951683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41824 (local address 127.4.159.131:42305)
0807 08:48:07.951737 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:07.951739 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:07.951748 (+ 9us) server_negotiation.cc:407] Connection header received
0807 08:48:07.951799 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:07.951800 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:07.951810 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:07.951853 (+ 43us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:48:07.951948 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:07.952422 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:07.953098 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:07.953297 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:07.953351 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:07.953473 (+ 122us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:48:07.953505 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:48:07.953544 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0807 08:48:07.953593 (+ 49us) server_negotiation.cc:299] Negotiation successful
0807 08:48:07.953615 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 08:48:07.957733 16643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:07.955805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.131:49589 (local address 127.4.159.129:41925)
0807 08:48:07.955866 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:07.955868 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:07.955932 (+ 64us) server_negotiation.cc:407] Connection header received
0807 08:48:07.955978 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:07.955978 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:07.955986 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:07.956026 (+ 40us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:48:07.956102 (+ 76us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:07.956592 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:07.957262 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:07.957431 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:07.957470 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:07.957584 (+ 114us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:48:07.957605 (+ 21us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:48:07.957634 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0807 08:48:07.957666 (+ 32us) server_negotiation.cc:299] Negotiation successful
0807 08:48:07.957679 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260807 08:48:07.957754 16796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:07.955751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.129:41925 (local address 127.4.159.131:49589)
0807 08:48:07.955882 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:07.955891 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:07.955941 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:07.956030 (+ 89us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:07.956030 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:07.956031 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 08:48:07.956070 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:07.956071 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:07.956648 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:07.956648 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:07.957204 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:07.957206 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:07.957480 (+ 274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:07.957480 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:07.957517 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:07.957530 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 08:48:07.957635 (+ 105us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 08:48:07.957636 (+ 1us) client_negotiation.cc:769] Sending connection context
0807 08:48:07.957663 (+ 27us) client_negotiation.cc:240] Negotiation successful
0807 08:48:07.957698 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":63,"threads_started":1}
I20260807 08:48:07.959427 16317 raft_consensus.cc:1261] T 2584a398d3f6437fbf76c21addd6ce1d P a6627f8faaf1423fb7504793451b18c3 [term 1 FOLLOWER]: Refusing update from remote peer 9bcc8a5781ff4eb5bcbe77690fd0c0d4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:48:07.959488 16097 raft_consensus.cc:1261] T 2584a398d3f6437fbf76c21addd6ce1d P 4f8ad558036e4bcbbace591712a92519 [term 1 FOLLOWER]: Refusing update from remote peer 9bcc8a5781ff4eb5bcbe77690fd0c0d4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:48:07.959668 16789 consensus_queue.cc:1035] T 2584a398d3f6437fbf76c21addd6ce1d P 9bcc8a5781ff4eb5bcbe77690fd0c0d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f8ad558036e4bcbbace591712a92519" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 41925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:48:07.959782 16679 consensus_queue.cc:1035] T 2584a398d3f6437fbf76c21addd6ce1d P 9bcc8a5781ff4eb5bcbe77690fd0c0d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6627f8faaf1423fb7504793451b18c3" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 44955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:48:07.965693 16646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:07.963922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.129:40657 (local address 127.4.159.131:42305)
0807 08:48:07.963953 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:07.963955 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:07.963962 (+ 7us) server_negotiation.cc:407] Connection header received
0807 08:48:07.964011 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:07.964012 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:07.964018 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:07.964057 (+ 39us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:48:07.964133 (+ 76us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:07.964546 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:07.965216 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:07.965396 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:07.965441 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:07.965519 (+ 78us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:48:07.965539 (+ 20us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:48:07.965568 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0807 08:48:07.965616 (+ 48us) server_negotiation.cc:299] Negotiation successful
0807 08:48:07.965633 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260807 08:48:07.965724 16741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:07.963883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.131:42305 (local address 127.4.159.129:40657)
0807 08:48:07.963926 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:07.963934 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:07.963967 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:07.964061 (+ 94us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:07.964061 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:07.964062 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 08:48:07.964093 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:07.964094 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:07.964601 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:07.964602 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:07.965158 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:07.965160 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:07.965445 (+ 285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:07.965445 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:07.965479 (+ 34us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:07.965484 (+ 5us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 08:48:07.965572 (+ 88us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 08:48:07.965574 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 08:48:07.965613 (+ 39us) client_negotiation.cc:240] Negotiation successful
0807 08:48:07.965655 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260807 08:48:07.972126 15852 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52364:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:48:07.972201 15852 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52364:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:48:07.972788 15852 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4ea7dd9dc02e498b932350ceaf4214cb: Sending DeleteTablet for 3 replicas of tablet 2584a398d3f6437fbf76c21addd6ce1d
I20260807 08:48:07.973006 16286 tablet_service.cc:1514] Processing DeleteTablet for tablet 2584a398d3f6437fbf76c21addd6ce1d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:48:07 UTC) from {username='oryx', principal='oryx/127.4.159.190@KRBTEST.COM'} at 127.0.0.1:60196
I20260807 08:48:07.973027 16504 tablet_service.cc:1514] Processing DeleteTablet for tablet 2584a398d3f6437fbf76c21addd6ce1d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:48:07 UTC) from {username='oryx', principal='oryx/127.4.159.190@KRBTEST.COM'} at 127.0.0.1:41800
I20260807 08:48:07.973096 16693 tablet_replica.cc:330] T 2584a398d3f6437fbf76c21addd6ce1d P a6627f8faaf1423fb7504793451b18c3: stopping tablet replica
I20260807 08:48:07.973114 16687 tablet_replica.cc:330] T 2584a398d3f6437fbf76c21addd6ce1d P 9bcc8a5781ff4eb5bcbe77690fd0c0d4: stopping tablet replica
I20260807 08:48:07.973140 16693 raft_consensus.cc:2229] T 2584a398d3f6437fbf76c21addd6ce1d P a6627f8faaf1423fb7504793451b18c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:48:07.973166 16687 raft_consensus.cc:2229] T 2584a398d3f6437fbf76c21addd6ce1d P 9bcc8a5781ff4eb5bcbe77690fd0c0d4 [term 1 LEADER]: Raft consensus shutting down.
I20260807 08:48:07.973170 16693 raft_consensus.cc:2258] T 2584a398d3f6437fbf76c21addd6ce1d P a6627f8faaf1423fb7504793451b18c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:48:07.973215 16687 raft_consensus.cc:2258] T 2584a398d3f6437fbf76c21addd6ce1d P 9bcc8a5781ff4eb5bcbe77690fd0c0d4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:48:07.973332 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 15966
I20260807 08:48:07.973366 16068 tablet_service.cc:1514] Processing DeleteTablet for tablet 2584a398d3f6437fbf76c21addd6ce1d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:48:07 UTC) from {username='oryx', principal='oryx/127.4.159.190@KRBTEST.COM'} at 127.0.0.1:57072
I20260807 08:48:07.973438 16688 tablet_replica.cc:330] T 2584a398d3f6437fbf76c21addd6ce1d P 4f8ad558036e4bcbbace591712a92519: stopping tablet replica
I20260807 08:48:07.973479 16688 raft_consensus.cc:2229] T 2584a398d3f6437fbf76c21addd6ce1d P 4f8ad558036e4bcbbace591712a92519 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:48:07.973486 16687 ts_tablet_manager.cc:1905] T 2584a398d3f6437fbf76c21addd6ce1d P 9bcc8a5781ff4eb5bcbe77690fd0c0d4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:48:07.973505 16688 raft_consensus.cc:2258] T 2584a398d3f6437fbf76c21addd6ce1d P 4f8ad558036e4bcbbace591712a92519 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:48:07.975806 16687 ts_tablet_manager.cc:1918] T 2584a398d3f6437fbf76c21addd6ce1d P 9bcc8a5781ff4eb5bcbe77690fd0c0d4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 08:48:07.975867 16687 log.cc:1196] T 2584a398d3f6437fbf76c21addd6ce1d P 9bcc8a5781ff4eb5bcbe77690fd0c0d4: Deleting WAL directory at /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786092453486433-4734-0/minicluster-data/ts-2/wal/wals/2584a398d3f6437fbf76c21addd6ce1d
I20260807 08:48:07.976048 16687 ts_tablet_manager.cc:1939] T 2584a398d3f6437fbf76c21addd6ce1d P 9bcc8a5781ff4eb5bcbe77690fd0c0d4: Deleting consensus metadata
I20260807 08:48:07.977766 16693 ts_tablet_manager.cc:1905] T 2584a398d3f6437fbf76c21addd6ce1d P a6627f8faaf1423fb7504793451b18c3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:48:07.978924 15829 catalog_manager.cc:4841] TS 9bcc8a5781ff4eb5bcbe77690fd0c0d4 (127.4.159.131:42305): tablet 2584a398d3f6437fbf76c21addd6ce1d (table test-table [id=7e54383015db4909b3a5bad4e4b1d372]) successfully deleted
I20260807 08:48:07.979133 16693 ts_tablet_manager.cc:1918] T 2584a398d3f6437fbf76c21addd6ce1d P a6627f8faaf1423fb7504793451b18c3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 08:48:07.979192 16693 log.cc:1196] T 2584a398d3f6437fbf76c21addd6ce1d P a6627f8faaf1423fb7504793451b18c3: Deleting WAL directory at /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786092453486433-4734-0/minicluster-data/ts-1/wal/wals/2584a398d3f6437fbf76c21addd6ce1d
I20260807 08:48:07.979334 16693 ts_tablet_manager.cc:1939] T 2584a398d3f6437fbf76c21addd6ce1d P a6627f8faaf1423fb7504793451b18c3: Deleting consensus metadata
I20260807 08:48:07.979691 15824 catalog_manager.cc:4841] TS a6627f8faaf1423fb7504793451b18c3 (127.4.159.130:44955): tablet 2584a398d3f6437fbf76c21addd6ce1d (table test-table [id=7e54383015db4909b3a5bad4e4b1d372]) successfully deleted
W20260807 08:48:07.984690 15826 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.159.129:41925 (error 108)
I20260807 08:48:07.984896 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 16181
I20260807 08:48:07.994927 16802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:07.984969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.129:41925 (local address 127.0.0.1:57098)
0807 08:48:07.994831 (+ 9862us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:07.994885 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.129:41925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":9844,"thread_start_us":38,"threads_started":1}
W20260807 08:48:07.995023 15826 catalog_manager.cc:4584] TS 4f8ad558036e4bcbbace591712a92519 (127.4.159.129:41925): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2584a398d3f6437fbf76c21addd6ce1d: Network error: Client connection negotiation failed: client connection to 127.4.159.129:41925: connect: Connection refused (error 111)
I20260807 08:48:07.999147 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 16393
I20260807 08:48:08.010682 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 15797
2026-08-07T08:48:08Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (2912 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786092453486433-4734-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:48:08 dist-test-slave-6zbq krb5kdc[16808](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:48:08 dist-test-slave-6zbq krb5kdc[16808](info): set up 2 sockets
Aug 07 08:48:08 dist-test-slave-6zbq krb5kdc[16808](info): commencing operation
krb5kdc: starting...
W20260807 08:48:09.125674 4734 mini_kdc.cc:121] Time spent starting KDC: real 1.089s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786092453486433-4734-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786092453486433-4734-0/krb5kdc/test-user.keytab.
Aug 07 08:48:09 dist-test-slave-6zbq krb5kdc[16808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092489, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:48:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:48:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.159.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.159.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.159.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab.
Entry for principal kudu/127.4.159.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab.
I20260807 08:48:09.231246 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786092453486433-4734-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786092453486433-4734-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786092453486433-4734-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.159.190:41787
--webserver_interface=127.4.159.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab
--principal=kudu/127.4.159.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:33393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.159.190:41787
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:48:09.309273 16827 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:48:09.309463 16827 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:48:09.309494 16827 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:48:09.310676 16827 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:48:09.310724 16827 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:48:09.310743 16827 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:48:09.310760 16827 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:48:09.310774 16827 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:48:09.312247 16827 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:33393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786092453486433-4734-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786092453486433-4734-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.159.190:41787
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.159.190:41787
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.159.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16827
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786092453486433-4734-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:48:09.312476 16827 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:48:09.312669 16827 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:48:09.317162 16840 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:48:09.317186 16837 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:48:09.317175 16836 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:48:09.317407 16827 server_base.cc:1034] running on GCE node
I20260807 08:48:09.317628 16827 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:48:09.317884 16827 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:48:09.319020 16827 hybrid_clock.cc:648] HybridClock initialized: now 1786092489319004 us; error 28 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-07T08:48:09Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1310 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:48:09 dist-test-slave-6zbq krb5kdc[16848](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:48:09 dist-test-slave-6zbq krb5kdc[16848](info): set up 2 sockets
Aug 07 08:48:09 dist-test-slave-6zbq krb5kdc[16848](info): commencing operation
krb5kdc: starting...
W20260807 08:48:10.447224 4734 mini_kdc.cc:121] Time spent starting KDC: real 1.099s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/krb5kdc/test-user.keytab.
Aug 07 08:48:10 dist-test-slave-6zbq krb5kdc[16848](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092490, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:48:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:48:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.159.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.159.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.159.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab.
Entry for principal kudu/127.4.159.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab.
I20260807 08:48:10.554706 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.159.190:43653
--webserver_interface=127.4.159.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab
--principal=kudu/127.4.159.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:45469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.159.190:43653
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:48:10.633862 16863 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:48:10.634045 16863 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:48:10.634075 16863 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:48:10.635227 16863 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:48:10.635267 16863 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:48:10.635286 16863 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:48:10.635303 16863 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:48:10.635318 16863 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:48:10.636763 16863 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:45469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.159.190:43653
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab
--principal=kudu/127.4.159.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.159.190:43653
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.159.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16863
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:48:10.636992 16863 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:48:10.637183 16863 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:48:10.641533 16877 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:48:10.641542 16874 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:48:10.641542 16871 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:48:10.641794 16863 server_base.cc:1034] running on GCE node
I20260807 08:48:10.642009 16863 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:48:10.642275 16863 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:48:10.643405 16863 hybrid_clock.cc:648] HybridClock initialized: now 1786092490643407 us; error 38 us; skew 500 ppm
Aug 07 08:48:10 dist-test-slave-6zbq krb5kdc[16848](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092490, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:48:10.649639 16863 init.cc:377] Logged in from keytab as kudu/127.4.159.190@KRBTEST.COM (short username kudu)
I20260807 08:48:10.650040 16863 webserver.cc:460] Webserver started at http://127.4.159.190:42995/ using document root <none> and password file <none>
I20260807 08:48:10.650236 16863 fs_manager.cc:359] Metadata directory not provided
I20260807 08:48:10.650283 16863 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:48:10.650426 16863 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:48:10.651467 16863 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/master-0/data/instance:
uuid: "31e175821f0142d5886ab45ecca3ae3d"
format_stamp: "Formatted at 2026-08-07 08:48:10 on dist-test-slave-6zbq"
server_key: "e2bb1b0524baffe8e426d43c0da77907"
server_key_iv: "4080f9398d4bd35545d1cf2f2de503e4"
server_key_version: "encryptionkey@0"
I20260807 08:48:10.651793 16863 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/master-0/wal/instance:
uuid: "31e175821f0142d5886ab45ecca3ae3d"
format_stamp: "Formatted at 2026-08-07 08:48:10 on dist-test-slave-6zbq"
server_key: "e2bb1b0524baffe8e426d43c0da77907"
server_key_iv: "4080f9398d4bd35545d1cf2f2de503e4"
server_key_version: "encryptionkey@0"
I20260807 08:48:10.652942 16863 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 08:48:10.653553 16891 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:48:10.653692 16863 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 08:48:10.653780 16863 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/master-0/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/master-0/wal
uuid: "31e175821f0142d5886ab45ecca3ae3d"
format_stamp: "Formatted at 2026-08-07 08:48:10 on dist-test-slave-6zbq"
server_key: "e2bb1b0524baffe8e426d43c0da77907"
server_key_iv: "4080f9398d4bd35545d1cf2f2de503e4"
server_key_version: "encryptionkey@0"
I20260807 08:48:10.653841 16863 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:48:10.675354 16863 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:48:10.677032 16863 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:48:10.677162 16863 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:48:10.680877 16863 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.190:43653
I20260807 08:48:10.680920 16976 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.190:43653 every 8 connection(s)
I20260807 08:48:10.681234 16863 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
I20260807 08:48:10.682009 16977 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:48:10.684154 16977 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31e175821f0142d5886ab45ecca3ae3d: Bootstrap starting.
I20260807 08:48:10.684660 16977 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 31e175821f0142d5886ab45ecca3ae3d: Neither blocks nor log segments found. Creating new log.
I20260807 08:48:10.684882 16977 log.cc:824] T 00000000000000000000000000000000 P 31e175821f0142d5886ab45ecca3ae3d: Log is configured to *not* fsync() on all Append() calls
I20260807 08:48:10.685469 16977 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31e175821f0142d5886ab45ecca3ae3d: No bootstrap required, opened a new log
I20260807 08:48:10.686997 16977 raft_consensus.cc:348] T 00000000000000000000000000000000 P 31e175821f0142d5886ab45ecca3ae3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31e175821f0142d5886ab45ecca3ae3d" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 43653 } }
I20260807 08:48:10.687141 16977 raft_consensus.cc:374] T 00000000000000000000000000000000 P 31e175821f0142d5886ab45ecca3ae3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:48:10.687172 16977 raft_consensus.cc:729] T 00000000000000000000000000000000 P 31e175821f0142d5886ab45ecca3ae3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31e175821f0142d5886ab45ecca3ae3d, State: Initialized, Role: FOLLOWER
I20260807 08:48:10.687280 16977 consensus_queue.cc:260] T 00000000000000000000000000000000 P 31e175821f0142d5886ab45ecca3ae3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31e175821f0142d5886ab45ecca3ae3d" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 43653 } }
I20260807 08:48:10.687356 16977 raft_consensus.cc:388] T 00000000000000000000000000000000 P 31e175821f0142d5886ab45ecca3ae3d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:48:10.687384 16977 raft_consensus.cc:482] T 00000000000000000000000000000000 P 31e175821f0142d5886ab45ecca3ae3d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:48:10.687417 16977 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 31e175821f0142d5886ab45ecca3ae3d [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:48:10.687844 16977 raft_consensus.cc:504] T 00000000000000000000000000000000 P 31e175821f0142d5886ab45ecca3ae3d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31e175821f0142d5886ab45ecca3ae3d" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 43653 } }
I20260807 08:48:10.687922 16977 leader_election.cc:302] T 00000000000000000000000000000000 P 31e175821f0142d5886ab45ecca3ae3d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 31e175821f0142d5886ab45ecca3ae3d; no voters:
I20260807 08:48:10.688035 16977 leader_election.cc:288] T 00000000000000000000000000000000 P 31e175821f0142d5886ab45ecca3ae3d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:48:10.688077 16981 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 31e175821f0142d5886ab45ecca3ae3d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:48:10.688230 16981 raft_consensus.cc:686] T 00000000000000000000000000000000 P 31e175821f0142d5886ab45ecca3ae3d [term 1 LEADER]: Becoming Leader. State: Replica: 31e175821f0142d5886ab45ecca3ae3d, State: Running, Role: LEADER
I20260807 08:48:10.688264 16977 sys_catalog.cc:564] T 00000000000000000000000000000000 P 31e175821f0142d5886ab45ecca3ae3d [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:48:10.688318 16981 consensus_queue.cc:237] T 00000000000000000000000000000000 P 31e175821f0142d5886ab45ecca3ae3d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31e175821f0142d5886ab45ecca3ae3d" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 43653 } }
I20260807 08:48:10.688587 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 16863
I20260807 08:48:10.688697 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/master-0/wal/instance
I20260807 08:48:10.688907 4734 external_mini_cluster.cc:1428] Setting key c891312f0e90d5c2ce0cfe16278d532d
I20260807 08:48:10.688870 16982 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31e175821f0142d5886ab45ecca3ae3d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "31e175821f0142d5886ab45ecca3ae3d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31e175821f0142d5886ab45ecca3ae3d" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 43653 } } }
I20260807 08:48:10.688943 16982 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31e175821f0142d5886ab45ecca3ae3d [sys.catalog]: This master's current role is: LEADER
I20260807 08:48:10.689095 16987 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31e175821f0142d5886ab45ecca3ae3d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 31e175821f0142d5886ab45ecca3ae3d. Latest consensus state: current_term: 1 leader_uuid: "31e175821f0142d5886ab45ecca3ae3d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31e175821f0142d5886ab45ecca3ae3d" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 43653 } } }
I20260807 08:48:10.689172 16987 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31e175821f0142d5886ab45ecca3ae3d [sys.catalog]: This master's current role is: LEADER
I20260807 08:48:10.689457 16997 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:48:10.690061 16997 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:48:10.691725 16997 catalog_manager.cc:1347] Generated new cluster ID: 2277a6df35d94f09930976a2ebda935f
I20260807 08:48:10.691774 16997 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 08:48:10 dist-test-slave-6zbq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092490, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.159.190@KRBTEST.COM
I20260807 08:48:10.699748 17006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:10.689328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33018 (local address 127.4.159.190:43653)
0807 08:48:10.689486 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:10.689495 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:10.689723 (+ 228us) server_negotiation.cc:407] Connection header received
0807 08:48:10.689881 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:10.689888 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:10.689946 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:10.690076 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:10.690819 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:10.691967 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:10.692989 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:10.693550 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:10.693648 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:10.696326 (+ 2678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:48:10.696330 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:48:10.696330 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:48:10.696342 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:48:10.698123 (+ 1781us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:48:10.698580 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:48:10.698582 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:48:10.698583 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:48:10.698694 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:48:10.698897 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:48:10.698898 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:48:10.698898 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:48:10.699160 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:48:10.699287 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0807 08:48:10.699413 (+ 126us) server_negotiation.cc:299] Negotiation successful
0807 08:48:10.699536 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":57,"threads_started":1}
I20260807 08:48:10.716894 16997 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:48:10.717615 16997 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:48:10.720407 16997 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 31e175821f0142d5886ab45ecca3ae3d: Generated new TSK 0
I20260807 08:48:10.720561 16997 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.159.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.159.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.159.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.129.keytab.
Entry for principal kudu/127.4.159.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.129.keytab.
I20260807 08:48:10.748751 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.159.129:0
--local_ip_for_outbound_sockets=127.4.159.129
--webserver_interface=127.4.159.129
--webserver_port=0
--tserver_master_addrs=127.4.159.190:43653
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.129.keytab
--principal=kudu/127.4.159.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:45469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:48:10.827409 17021 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:48:10.827587 17021 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:48:10.827607 17021 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:48:10.828698 17021 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:48:10.828737 17021 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:48:10.828765 17021 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.159.129
I20260807 08:48:10.830150 17021 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:45469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.129.keytab
--principal=kudu/127.4.159.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.159.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.159.129
--webserver_port=0
--tserver_master_addrs=127.4.159.190:43653
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17021
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.159.129
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:48:10.830386 17021 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:48:10.830587 17021 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:48:10.834972 17036 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:48:10.835011 17034 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:48:10.834982 17030 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:48:10.835188 17021 server_base.cc:1034] running on GCE node
I20260807 08:48:10.835381 17021 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:48:10.835594 17021 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:48:10.836731 17021 hybrid_clock.cc:648] HybridClock initialized: now 1786092490836724 us; error 26 us; skew 500 ppm
Aug 07 08:48:10 dist-test-slave-6zbq krb5kdc[16848](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092490, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:48:10.838969 17021 init.cc:377] Logged in from keytab as kudu/127.4.159.129@KRBTEST.COM (short username kudu)
I20260807 08:48:10.839334 17021 webserver.cc:460] Webserver started at http://127.4.159.129:34137/ using document root <none> and password file <none>
I20260807 08:48:10.839516 17021 fs_manager.cc:359] Metadata directory not provided
I20260807 08:48:10.839552 17021 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:48:10.839635 17021 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:48:10.840546 17021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/ts-0/data/instance:
uuid: "9011e86742cb445c9e1c04adaf434d5a"
format_stamp: "Formatted at 2026-08-07 08:48:10 on dist-test-slave-6zbq"
server_key: "9253f0f69cfd9617cc616f5eccf10a2a"
server_key_iv: "ac4fa51e77e641e1b5a5a800e9247673"
server_key_version: "encryptionkey@0"
I20260807 08:48:10.840816 17021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/ts-0/wal/instance:
uuid: "9011e86742cb445c9e1c04adaf434d5a"
format_stamp: "Formatted at 2026-08-07 08:48:10 on dist-test-slave-6zbq"
server_key: "9253f0f69cfd9617cc616f5eccf10a2a"
server_key_iv: "ac4fa51e77e641e1b5a5a800e9247673"
server_key_version: "encryptionkey@0"
I20260807 08:48:10.842056 17021 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 08:48:10.842705 17046 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:48:10.842856 17021 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 08:48:10.842937 17021 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/ts-0/wal
uuid: "9011e86742cb445c9e1c04adaf434d5a"
format_stamp: "Formatted at 2026-08-07 08:48:10 on dist-test-slave-6zbq"
server_key: "9253f0f69cfd9617cc616f5eccf10a2a"
server_key_iv: "ac4fa51e77e641e1b5a5a800e9247673"
server_key_version: "encryptionkey@0"
I20260807 08:48:10.843003 17021 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:48:10.856223 17021 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:48:10.857807 17021 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:48:10.857928 17021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:48:10.858105 17021 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:48:10.858414 17021 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:48:10.858455 17021 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:48:10.858482 17021 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:48:10.858502 17021 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:48:10.863813 17021 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.129:39751
I20260807 08:48:10.863873 17222 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.129:39751 every 8 connection(s)
I20260807 08:48:10.864121 17021 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/ts-0/data/info.pb
Aug 07 08:48:10 dist-test-slave-6zbq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092490, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.129@KRBTEST.COM for kudu/127.4.159.190@KRBTEST.COM
I20260807 08:48:10.872313 17006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:10.864772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.129:56883 (local address 127.4.159.190:43653)
0807 08:48:10.864818 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:10.864820 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:10.865002 (+ 182us) server_negotiation.cc:407] Connection header received
0807 08:48:10.865449 (+ 447us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:10.865450 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:10.865462 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:10.865513 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:10.866376 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:10.866871 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:10.867876 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:10.868085 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:10.868149 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:10.869990 (+ 1841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:48:10.869992 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:48:10.869992 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:48:10.869999 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:48:10.871107 (+ 1108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:48:10.871459 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:48:10.871461 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:48:10.871461 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:48:10.871514 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:48:10.871725 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:48:10.871726 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:48:10.871726 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:48:10.871831 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:48:10.871886 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 08:48:10.872134 (+ 248us) server_negotiation.cc:299] Negotiation successful
0807 08:48:10.872225 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 08:48:10.872450 17225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:10.864732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:43653 (local address 127.4.159.129:56883)
0807 08:48:10.864899 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:10.864929 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:10.865352 (+ 423us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:10.865576 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:10.865580 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:10.866093 (+ 513us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:48:10.866261 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:10.866265 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:10.866950 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:10.866952 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:10.867786 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:10.867789 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:10.868156 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:10.868157 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:10.868289 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:10.868699 (+ 410us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:48:10.868707 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:48:10.869903 (+ 1196us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:48:10.871189 (+ 1286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:48:10.871196 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:48:10.871198 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:48:10.871396 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:48:10.871566 (+ 170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:48:10.871567 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:48:10.871567 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:48:10.871671 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:48:10.871884 (+ 213us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:48:10.871890 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:48:10.872012 (+ 122us) client_negotiation.cc:769] Sending connection context
0807 08:48:10.872094 (+ 82us) client_negotiation.cc:240] Negotiation successful
0807 08:48:10.872258 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":91,"threads_started":1}
I20260807 08:48:10.872870 17223 heartbeater.cc:347] Connected to a master server at 127.4.159.190:43653
I20260807 08:48:10.872951 17223 heartbeater.cc:464] Registering TS with master...
I20260807 08:48:10.873104 17223 heartbeater.cc:511] Master 127.4.159.190:43653 requested a full tablet report, sending...
I20260807 08:48:10.873134 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 17021
I20260807 08:48:10.873232 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/ts-0/wal/instance
I20260807 08:48:10.873442 4734 external_mini_cluster.cc:1428] Setting key b879dadcb6d7bc3de64b4574e6db2000
I20260807 08:48:10.873633 16921 ts_manager.cc:194] Registered new tserver with Master: 9011e86742cb445c9e1c04adaf434d5a (127.4.159.129:39751)
I20260807 08:48:10.874385 16921 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.159.129@KRBTEST.COM'} at 127.4.159.129:56883
WARNING: no policy specified for kudu/127.4.159.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.159.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.159.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.130.keytab.
Entry for principal kudu/127.4.159.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.130.keytab.
I20260807 08:48:10.896277 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.159.130:0
--local_ip_for_outbound_sockets=127.4.159.130
--webserver_interface=127.4.159.130
--webserver_port=0
--tserver_master_addrs=127.4.159.190:43653
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.130.keytab
--principal=kudu/127.4.159.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:45469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:48:10.975432 17228 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:48:10.975625 17228 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:48:10.975646 17228 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:48:10.976727 17228 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:48:10.976759 17228 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:48:10.976783 17228 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.159.130
I20260807 08:48:10.978133 17228 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:45469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.130.keytab
--principal=kudu/127.4.159.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.159.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.159.130
--webserver_port=0
--tserver_master_addrs=127.4.159.190:43653
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17228
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.159.130
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:48:10.978360 17228 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:48:10.978544 17228 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:48:10.983119 17243 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:48:10.983167 17235 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:48:10.983135 17236 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:48:10.983322 17228 server_base.cc:1034] running on GCE node
I20260807 08:48:10.983505 17228 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:48:10.983713 17228 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:48:10.984865 17228 hybrid_clock.cc:648] HybridClock initialized: now 1786092490984852 us; error 28 us; skew 500 ppm
Aug 07 08:48:10 dist-test-slave-6zbq krb5kdc[16848](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092490, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:48:10.987454 17228 init.cc:377] Logged in from keytab as kudu/127.4.159.130@KRBTEST.COM (short username kudu)
I20260807 08:48:10.987840 17228 webserver.cc:460] Webserver started at http://127.4.159.130:35907/ using document root <none> and password file <none>
I20260807 08:48:10.988047 17228 fs_manager.cc:359] Metadata directory not provided
I20260807 08:48:10.988093 17228 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:48:10.988215 17228 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:48:10.989243 17228 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/ts-1/data/instance:
uuid: "c8e68df9c3e5491b985835084812c821"
format_stamp: "Formatted at 2026-08-07 08:48:10 on dist-test-slave-6zbq"
server_key: "3cbd4b4c44e1e548a761a273c6142175"
server_key_iv: "f4ed6665c513d76a6f6a514cd6df2efa"
server_key_version: "encryptionkey@0"
I20260807 08:48:10.989606 17228 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/ts-1/wal/instance:
uuid: "c8e68df9c3e5491b985835084812c821"
format_stamp: "Formatted at 2026-08-07 08:48:10 on dist-test-slave-6zbq"
server_key: "3cbd4b4c44e1e548a761a273c6142175"
server_key_iv: "f4ed6665c513d76a6f6a514cd6df2efa"
server_key_version: "encryptionkey@0"
I20260807 08:48:10.990904 17228 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 08:48:10.991549 17253 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:48:10.991705 17228 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 08:48:10.991786 17228 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/ts-1/wal
uuid: "c8e68df9c3e5491b985835084812c821"
format_stamp: "Formatted at 2026-08-07 08:48:10 on dist-test-slave-6zbq"
server_key: "3cbd4b4c44e1e548a761a273c6142175"
server_key_iv: "f4ed6665c513d76a6f6a514cd6df2efa"
server_key_version: "encryptionkey@0"
I20260807 08:48:10.991845 17228 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:48:11.001013 17228 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:48:11.002656 17228 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:48:11.002779 17228 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:48:11.002959 17228 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:48:11.003216 17228 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:48:11.003255 17228 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:48:11.003280 17228 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:48:11.003304 17228 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:48:11.008512 17228 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.130:33879
I20260807 08:48:11.008567 17440 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.130:33879 every 8 connection(s)
I20260807 08:48:11.008817 17228 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/ts-1/data/info.pb
I20260807 08:48:11.010514 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 17228
I20260807 08:48:11.010609 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/ts-1/wal/instance
I20260807 08:48:11.010783 4734 external_mini_cluster.cc:1428] Setting key 169761666ecbcf628d4b8859ec3e0b5f
Aug 07 08:48:11 dist-test-slave-6zbq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092490, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.130@KRBTEST.COM for kudu/127.4.159.190@KRBTEST.COM
I20260807 08:48:11.018554 17006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:11.009444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.130:58359 (local address 127.4.159.190:43653)
0807 08:48:11.009503 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:11.009505 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:11.009652 (+ 147us) server_negotiation.cc:407] Connection header received
0807 08:48:11.010076 (+ 424us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:11.010077 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:11.010086 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:11.010152 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:11.011188 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:11.011773 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:11.013271 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:11.013572 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:11.013650 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:11.015872 (+ 2222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:48:11.015874 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:48:11.015874 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:48:11.015881 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:48:11.017183 (+ 1302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:48:11.017638 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:48:11.017638 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:48:11.017638 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:48:11.017683 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:48:11.017933 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:48:11.017933 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:48:11.017934 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:48:11.018027 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:48:11.018070 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0807 08:48:11.018420 (+ 350us) server_negotiation.cc:299] Negotiation successful
0807 08:48:11.018488 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260807 08:48:11.018800 17443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:11.009402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:43653 (local address 127.4.159.130:58359)
0807 08:48:11.009542 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:11.009579 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:11.010005 (+ 426us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:11.010190 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:11.010193 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:11.010864 (+ 671us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:48:11.011086 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:11.011093 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:11.011864 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:11.011865 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:11.012969 (+ 1104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:11.013054 (+ 85us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:11.013659 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:11.013660 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:11.013788 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:11.014284 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:48:11.014293 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:48:11.015770 (+ 1477us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:48:11.017263 (+ 1493us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:48:11.017270 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:48:11.017273 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:48:11.017559 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:48:11.017737 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:48:11.017738 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:48:11.017739 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:48:11.017875 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:48:11.018079 (+ 204us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:48:11.018085 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:48:11.018255 (+ 170us) client_negotiation.cc:769] Sending connection context
0807 08:48:11.018407 (+ 152us) client_negotiation.cc:240] Negotiation successful
0807 08:48:11.018579 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":76,"threads_started":1}
I20260807 08:48:11.019305 17441 heartbeater.cc:347] Connected to a master server at 127.4.159.190:43653
I20260807 08:48:11.019393 17441 heartbeater.cc:464] Registering TS with master...
I20260807 08:48:11.019559 17441 heartbeater.cc:511] Master 127.4.159.190:43653 requested a full tablet report, sending...
I20260807 08:48:11.019912 16921 ts_manager.cc:194] Registered new tserver with Master: c8e68df9c3e5491b985835084812c821 (127.4.159.130:33879)
I20260807 08:48:11.020269 16921 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.159.130@KRBTEST.COM'} at 127.4.159.130:58359
WARNING: no policy specified for kudu/127.4.159.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.159.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.159.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.131.keytab.
Entry for principal kudu/127.4.159.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.131.keytab.
I20260807 08:48:11.034925 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.159.131:0
--local_ip_for_outbound_sockets=127.4.159.131
--webserver_interface=127.4.159.131
--webserver_port=0
--tserver_master_addrs=127.4.159.190:43653
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.131.keytab
--principal=kudu/127.4.159.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:45469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:48:11.113258 17446 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:48:11.113438 17446 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:48:11.113468 17446 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:48:11.114604 17446 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:48:11.114646 17446 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:48:11.114688 17446 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.159.131
I20260807 08:48:11.116060 17446 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:45469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.131.keytab
--principal=kudu/127.4.159.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.159.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.159.131
--webserver_port=0
--tserver_master_addrs=127.4.159.190:43653
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17446
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.159.131
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:48:11.116267 17446 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:48:11.116456 17446 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:48:11.120954 17454 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:48:11.121001 17446 server_base.cc:1034] running on GCE node
W20260807 08:48:11.120960 17453 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:48:11.120982 17457 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:48:11.121404 17446 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:48:11.121589 17446 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:48:11.122712 17446 hybrid_clock.cc:648] HybridClock initialized: now 1786092491122700 us; error 24 us; skew 500 ppm
Aug 07 08:48:11 dist-test-slave-6zbq krb5kdc[16848](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092491, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:48:11.124788 17446 init.cc:377] Logged in from keytab as kudu/127.4.159.131@KRBTEST.COM (short username kudu)
I20260807 08:48:11.125154 17446 webserver.cc:460] Webserver started at http://127.4.159.131:42627/ using document root <none> and password file <none>
I20260807 08:48:11.125339 17446 fs_manager.cc:359] Metadata directory not provided
I20260807 08:48:11.125386 17446 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:48:11.125507 17446 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:48:11.126487 17446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/ts-2/data/instance:
uuid: "69851855ca6d4c6680181bf785c8b02c"
format_stamp: "Formatted at 2026-08-07 08:48:11 on dist-test-slave-6zbq"
server_key: "794737083233a9af74d55b23a9268273"
server_key_iv: "17d189d32b1e8bb2d5fa3cd7f4ec2cca"
server_key_version: "encryptionkey@0"
I20260807 08:48:11.126849 17446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/ts-2/wal/instance:
uuid: "69851855ca6d4c6680181bf785c8b02c"
format_stamp: "Formatted at 2026-08-07 08:48:11 on dist-test-slave-6zbq"
server_key: "794737083233a9af74d55b23a9268273"
server_key_iv: "17d189d32b1e8bb2d5fa3cd7f4ec2cca"
server_key_version: "encryptionkey@0"
I20260807 08:48:11.128005 17446 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 08:48:11.128643 17469 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:48:11.128803 17446 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 08:48:11.128881 17446 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/ts-2/wal
uuid: "69851855ca6d4c6680181bf785c8b02c"
format_stamp: "Formatted at 2026-08-07 08:48:11 on dist-test-slave-6zbq"
server_key: "794737083233a9af74d55b23a9268273"
server_key_iv: "17d189d32b1e8bb2d5fa3cd7f4ec2cca"
server_key_version: "encryptionkey@0"
I20260807 08:48:11.128945 17446 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:48:11.144513 17446 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:48:11.146273 17446 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:48:11.146420 17446 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:48:11.146600 17446 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:48:11.146878 17446 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:48:11.146911 17446 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:48:11.146929 17446 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:48:11.146951 17446 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:48:11.152066 17446 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.131:41775
I20260807 08:48:11.152100 17663 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.131:41775 every 8 connection(s)
I20260807 08:48:11.152356 17446 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/ts-2/data/info.pb
I20260807 08:48:11.159076 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 17446
I20260807 08:48:11.159199 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/ts-2/wal/instance
I20260807 08:48:11.159379 4734 external_mini_cluster.cc:1428] Setting key 536d1d22181983855eff7109830ca859
Aug 07 08:48:11 dist-test-slave-6zbq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092491, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.131@KRBTEST.COM for kudu/127.4.159.190@KRBTEST.COM
I20260807 08:48:11.163488 17006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:11.153204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.131:49579 (local address 127.4.159.190:43653)
0807 08:48:11.153248 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:11.153250 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:11.153261 (+ 11us) server_negotiation.cc:407] Connection header received
0807 08:48:11.153859 (+ 598us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:11.153861 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:11.153875 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:11.153954 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:11.155164 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:11.155903 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:11.157375 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:11.157675 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:11.157739 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:11.160362 (+ 2623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:48:11.160364 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:48:11.160364 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:48:11.160373 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:48:11.161951 (+ 1578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:48:11.162321 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:48:11.162323 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:48:11.162323 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:48:11.162421 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:48:11.162788 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:48:11.162789 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:48:11.162790 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:48:11.162936 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:48:11.162982 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0807 08:48:11.163324 (+ 342us) server_negotiation.cc:299] Negotiation successful
0807 08:48:11.163402 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260807 08:48:11.163894 17666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:11.152936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:43653 (local address 127.4.159.131:49579)
0807 08:48:11.153093 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:11.153166 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:11.153737 (+ 571us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:11.154032 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:11.154036 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:11.154790 (+ 754us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:48:11.155050 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:11.155056 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:11.156013 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:11.156014 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:11.157292 (+ 1278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:11.157295 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:11.157769 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:11.157770 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:11.157926 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:11.158421 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:48:11.158431 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:48:11.160180 (+ 1749us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:48:11.162033 (+ 1853us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:48:11.162039 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:48:11.162041 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:48:11.162243 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:48:11.162511 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:48:11.162512 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:48:11.162513 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:48:11.162693 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:48:11.163032 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:48:11.163038 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:48:11.163187 (+ 149us) client_negotiation.cc:769] Sending connection context
0807 08:48:11.163286 (+ 99us) client_negotiation.cc:240] Negotiation successful
0807 08:48:11.163433 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":85,"threads_started":1}
I20260807 08:48:11.164525 17664 heartbeater.cc:347] Connected to a master server at 127.4.159.190:43653
I20260807 08:48:11.164633 17664 heartbeater.cc:464] Registering TS with master...
I20260807 08:48:11.164808 17664 heartbeater.cc:511] Master 127.4.159.190:43653 requested a full tablet report, sending...
I20260807 08:48:11.165194 16921 ts_manager.cc:194] Registered new tserver with Master: 69851855ca6d4c6680181bf785c8b02c (127.4.159.131:41775)
I20260807 08:48:11.165565 16921 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.159.131@KRBTEST.COM'} at 127.4.159.131:49579
I20260807 08:48:11.173875 4734 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.4.159.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.159.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.159.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/krb5kdc/oryx_127.4.159.129.keytab.
Entry for principal oryx/127.4.159.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/krb5kdc/oryx_127.4.159.129.keytab.
WARNING: no policy specified for oryx/127.4.159.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.159.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.159.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/krb5kdc/oryx_127.4.159.130.keytab.
Entry for principal oryx/127.4.159.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/krb5kdc/oryx_127.4.159.130.keytab.
WARNING: no policy specified for oryx/127.4.159.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.159.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.159.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/krb5kdc/oryx_127.4.159.131.keytab.
Entry for principal oryx/127.4.159.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/krb5kdc/oryx_127.4.159.131.keytab.
I20260807 08:48:11.229616 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 17021
I20260807 08:48:11.243407 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 17228
I20260807 08:48:11.249022 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 17446
I20260807 08:48:11.255947 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 16863
I20260807 08:48:11.268587 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.159.190:43653
--webserver_interface=127.4.159.190
--webserver_port=42995
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab
--principal=kudu/127.4.159.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:45469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.159.190:43653
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:48:11.346494 17677 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:48:11.346675 17677 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:48:11.346695 17677 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:48:11.347747 17677 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:48:11.347780 17677 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:48:11.347792 17677 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:48:11.347803 17677 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:48:11.347813 17677 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:48:11.349172 17677 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:45469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.159.190:43653
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab
--principal=kudu/127.4.159.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.159.190:43653
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.159.190
--webserver_port=42995
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17677
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:48:11.349350 17677 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:48:11.349510 17677 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:48:11.353914 17684 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:48:11.353940 17683 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:48:11.353914 17686 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:48:11.354153 17677 server_base.cc:1034] running on GCE node
I20260807 08:48:11.354398 17677 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:48:11.354650 17677 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:48:11.355783 17677 hybrid_clock.cc:648] HybridClock initialized: now 1786092491355770 us; error 27 us; skew 500 ppm
Aug 07 08:48:11 dist-test-slave-6zbq krb5kdc[16848](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092491, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:48:11.358120 17677 init.cc:377] Logged in from keytab as kudu/127.4.159.190@KRBTEST.COM (short username kudu)
I20260807 08:48:11.358553 17677 webserver.cc:460] Webserver started at http://127.4.159.190:42995/ using document root <none> and password file <none>
I20260807 08:48:11.358736 17677 fs_manager.cc:359] Metadata directory not provided
I20260807 08:48:11.358793 17677 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:48:11.360028 17677 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:48:11.360603 17697 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:48:11.360760 17677 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 08:48:11.360841 17677 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/master-0/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/master-0/wal
uuid: "31e175821f0142d5886ab45ecca3ae3d"
format_stamp: "Formatted at 2026-08-07 08:48:10 on dist-test-slave-6zbq"
server_key: "e2bb1b0524baffe8e426d43c0da77907"
server_key_iv: "4080f9398d4bd35545d1cf2f2de503e4"
server_key_version: "encryptionkey@0"
I20260807 08:48:11.361122 17677 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:48:11.385102 17677 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:48:11.386626 17677 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:48:11.386731 17677 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:48:11.390210 17677 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.190:43653
I20260807 08:48:11.390247 17774 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.190:43653 every 8 connection(s)
I20260807 08:48:11.390568 17677 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
I20260807 08:48:11.391873 17775 sys_catalog.cc:263] Verifying existing consensus state
I20260807 08:48:11.392480 17775 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31e175821f0142d5886ab45ecca3ae3d: Bootstrap starting.
I20260807 08:48:11.392807 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 17677
I20260807 08:48:11.393049 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.159.129:39751
--local_ip_for_outbound_sockets=127.4.159.129
--tserver_master_addrs=127.4.159.190:43653
--webserver_port=34137
--webserver_interface=127.4.159.129
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.129.keytab
--principal=kudu/127.4.159.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:45469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.4.159.129
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/krb5kdc/oryx_127.4.159.129.keytab with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 08:48:11.401075 17775 log.cc:824] T 00000000000000000000000000000000 P 31e175821f0142d5886ab45ecca3ae3d: Log is configured to *not* fsync() on all Append() calls
I20260807 08:48:11.402587 17775 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31e175821f0142d5886ab45ecca3ae3d: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 08:48:11.402824 17775 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31e175821f0142d5886ab45ecca3ae3d: Bootstrap complete.
I20260807 08:48:11.404791 17775 raft_consensus.cc:348] T 00000000000000000000000000000000 P 31e175821f0142d5886ab45ecca3ae3d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31e175821f0142d5886ab45ecca3ae3d" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 43653 } }
I20260807 08:48:11.405001 17775 raft_consensus.cc:729] T 00000000000000000000000000000000 P 31e175821f0142d5886ab45ecca3ae3d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31e175821f0142d5886ab45ecca3ae3d, State: Initialized, Role: FOLLOWER
I20260807 08:48:11.405138 17775 consensus_queue.cc:260] T 00000000000000000000000000000000 P 31e175821f0142d5886ab45ecca3ae3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31e175821f0142d5886ab45ecca3ae3d" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 43653 } }
I20260807 08:48:11.405211 17775 raft_consensus.cc:388] T 00000000000000000000000000000000 P 31e175821f0142d5886ab45ecca3ae3d [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:48:11.405260 17775 raft_consensus.cc:482] T 00000000000000000000000000000000 P 31e175821f0142d5886ab45ecca3ae3d [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:48:11.405305 17775 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 31e175821f0142d5886ab45ecca3ae3d [term 1 FOLLOWER]: Advancing to term 2
I20260807 08:48:11.406206 17775 raft_consensus.cc:504] T 00000000000000000000000000000000 P 31e175821f0142d5886ab45ecca3ae3d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31e175821f0142d5886ab45ecca3ae3d" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 43653 } }
I20260807 08:48:11.406299 17775 leader_election.cc:302] T 00000000000000000000000000000000 P 31e175821f0142d5886ab45ecca3ae3d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 31e175821f0142d5886ab45ecca3ae3d; no voters:
I20260807 08:48:11.406466 17775 leader_election.cc:288] T 00000000000000000000000000000000 P 31e175821f0142d5886ab45ecca3ae3d [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 08:48:11.406515 17779 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 31e175821f0142d5886ab45ecca3ae3d [term 2 FOLLOWER]: Leader election won for term 2
I20260807 08:48:11.406632 17779 raft_consensus.cc:686] T 00000000000000000000000000000000 P 31e175821f0142d5886ab45ecca3ae3d [term 2 LEADER]: Becoming Leader. State: Replica: 31e175821f0142d5886ab45ecca3ae3d, State: Running, Role: LEADER
I20260807 08:48:11.406720 17779 consensus_queue.cc:237] T 00000000000000000000000000000000 P 31e175821f0142d5886ab45ecca3ae3d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31e175821f0142d5886ab45ecca3ae3d" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 43653 } }
I20260807 08:48:11.406749 17775 sys_catalog.cc:564] T 00000000000000000000000000000000 P 31e175821f0142d5886ab45ecca3ae3d [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:48:11.407021 17783 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31e175821f0142d5886ab45ecca3ae3d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "31e175821f0142d5886ab45ecca3ae3d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31e175821f0142d5886ab45ecca3ae3d" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 43653 } } }
I20260807 08:48:11.407100 17783 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31e175821f0142d5886ab45ecca3ae3d [sys.catalog]: This master's current role is: LEADER
I20260807 08:48:11.407338 17784 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31e175821f0142d5886ab45ecca3ae3d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 31e175821f0142d5886ab45ecca3ae3d. Latest consensus state: current_term: 2 leader_uuid: "31e175821f0142d5886ab45ecca3ae3d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31e175821f0142d5886ab45ecca3ae3d" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 43653 } } }
I20260807 08:48:11.407410 17784 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31e175821f0142d5886ab45ecca3ae3d [sys.catalog]: This master's current role is: LEADER
I20260807 08:48:11.408398 17805 catalog_manager.cc:1259] Loaded cluster ID: 2277a6df35d94f09930976a2ebda935f
I20260807 08:48:11.408447 17805 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 31e175821f0142d5886ab45ecca3ae3d: loading cluster ID for follower catalog manager: success
I20260807 08:48:11.409560 17805 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 31e175821f0142d5886ab45ecca3ae3d: acquiring CA information for follower catalog manager: success
I20260807 08:48:11.409883 17805 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 31e175821f0142d5886ab45ecca3ae3d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 08:48:11.410460 17791 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:48:11.410647 17791 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:48:11.410734 17791 catalog_manager.cc:1259] Loaded cluster ID: 2277a6df35d94f09930976a2ebda935f
I20260807 08:48:11.410768 17791 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:48:11.410934 17791 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:48:11.411027 17791 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 31e175821f0142d5886ab45ecca3ae3d: Loaded TSK: 0
I20260807 08:48:11.411176 17791 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 08:48:11.472674 17777 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:48:11.472854 17777 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:48:11.472884 17777 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:48:11.473989 17777 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:48:11.474025 17777 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:48:11.474061 17777 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.159.129
I20260807 08:48:11.475461 17777 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:45469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/krb5kdc/oryx_127.4.159.129.keytab
--principal=oryx/127.4.159.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.159.129:39751
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.159.129
--webserver_port=34137
--tserver_master_addrs=127.4.159.190:43653
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17777
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.159.129
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:48:11.475679 17777 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:48:11.475869 17777 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 08:48:11.480408 17777 server_base.cc:1034] running on GCE node
W20260807 08:48:11.480561 17824 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:48:11.480580 17818 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:48:11.480597 17821 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:48:11.480962 17777 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:48:11.481266 17777 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:48:11.482424 17777 hybrid_clock.cc:648] HybridClock initialized: now 1786092491482399 us; error 38 us; skew 500 ppm
Aug 07 08:48:11 dist-test-slave-6zbq krb5kdc[16848](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092491, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.159.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:48:11.485306 17777 init.cc:377] Logged in from keytab as oryx/127.4.159.129@KRBTEST.COM (short username oryx)
I20260807 08:48:11.485828 17777 webserver.cc:460] Webserver started at http://127.4.159.129:34137/ using document root <none> and password file <none>
I20260807 08:48:11.486066 17777 fs_manager.cc:359] Metadata directory not provided
I20260807 08:48:11.486161 17777 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:48:11.487914 17777 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:48:11.488559 17834 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:48:11.488719 17777 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 08:48:11.488804 17777 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/ts-0/wal
uuid: "9011e86742cb445c9e1c04adaf434d5a"
format_stamp: "Formatted at 2026-08-07 08:48:10 on dist-test-slave-6zbq"
server_key: "9253f0f69cfd9617cc616f5eccf10a2a"
server_key_iv: "ac4fa51e77e641e1b5a5a800e9247673"
server_key_version: "encryptionkey@0"
I20260807 08:48:11.489099 17777 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:48:11.495867 17777 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:48:11.497730 17777 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:48:11.497836 17777 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:48:11.498054 17777 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:48:11.498327 17777 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:48:11.498407 17777 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:48:11.498441 17777 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:48:11.498464 17777 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:48:11.503705 17777 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.129:39751
I20260807 08:48:11.503842 18023 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.129:39751 every 8 connection(s)
I20260807 08:48:11.504139 17777 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/ts-0/data/info.pb
I20260807 08:48:11.508257 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 17777
I20260807 08:48:11.508523 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.159.130:33879
--local_ip_for_outbound_sockets=127.4.159.130
--tserver_master_addrs=127.4.159.190:43653
--webserver_port=35907
--webserver_interface=127.4.159.130
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.130.keytab
--principal=kudu/127.4.159.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:45469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.4.159.130
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/krb5kdc/oryx_127.4.159.130.keytab with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 08:48:11 dist-test-slave-6zbq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.159.129@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:48:11 dist-test-slave-6zbq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.159.129@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:48:11.513429 18028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:11.505032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:43653 (local address 127.4.159.129:53469)
0807 08:48:11.505298 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:11.505336 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:11.505897 (+ 561us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:11.506579 (+ 682us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:11.506583 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:11.507092 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:48:11.507304 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:11.507309 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:11.509004 (+ 1695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:11.509005 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:11.510108 (+ 1103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:11.510112 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:11.510745 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:11.510747 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:11.510867 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:11.511272 (+ 405us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:48:11.511281 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:48:11.513249 (+ 1968us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.159.190:43653: Server oryx/127.4.159.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":100,"threads_started":1}
W20260807 08:48:11.513669 18032 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:48:11.505355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.129:53469 (local address 127.4.159.190:43653)
0807 08:48:11.506103 (+ 748us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:11.506112 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:11.506158 (+ 46us) server_negotiation.cc:407] Connection header received
0807 08:48:11.506277 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:11.506284 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:11.506361 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:11.506529 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:11.507413 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:11.508880 (+ 1467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:11.510205 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:11.510658 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:11.510753 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:11.513372 (+ 2619us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.159.129:53469: BlockingRecv error: recv got EOF from 127.4.159.129:53469 (error 108)
Metrics: {"server-negotiator.queue_time_us":692,"thread_start_us":85,"threads_started":1}
W20260807 08:48:11.513726 18024 heartbeater.cc:644] Failed to heartbeat to 127.4.159.190:43653 (0 consecutive failures): Not authorized: Failed to ping master at 127.4.159.190:43653: Client connection negotiation failed: client connection to 127.4.159.190:43653: Server oryx/127.4.159.190@KRBTEST.COM not found in Kerberos database
Aug 07 08:48:11 dist-test-slave-6zbq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.159.129@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:48:11 dist-test-slave-6zbq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.159.129@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:48:11.517583 18028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:11.513928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:43653 (local address 127.4.159.129:34745)
0807 08:48:11.513965 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:11.513976 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:11.514019 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:11.514169 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:11.514170 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:11.514291 (+ 121us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:48:11.514332 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:11.514333 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:11.514918 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:11.514919 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:11.515401 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:11.515403 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:11.515716 (+ 313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:11.515717 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:11.515774 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:11.516095 (+ 321us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:48:11.516100 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:48:11.517528 (+ 1428us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.159.190:43653: Server oryx/127.4.159.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":22}
W20260807 08:48:11.517663 18032 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:48:11.514042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.129:34745 (local address 127.4.159.190:43653)
0807 08:48:11.514079 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:11.514080 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:11.514087 (+ 7us) server_negotiation.cc:407] Connection header received
0807 08:48:11.514104 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:11.514105 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:11.514113 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:11.514156 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:11.514415 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:11.514857 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:11.515470 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:11.515662 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:11.515720 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:11.517612 (+ 1892us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.159.129:34745: BlockingRecv error: recv got EOF from 127.4.159.129:34745 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
Aug 07 08:48:11 dist-test-slave-6zbq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.159.129@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:48:11 dist-test-slave-6zbq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.159.129@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:48:11.521448 18028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:11.517951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:43653 (local address 127.4.159.129:59931)
0807 08:48:11.518033 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:11.518043 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:11.518084 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:11.518199 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:11.518199 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:11.518312 (+ 113us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:48:11.518335 (+ 23us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:11.518336 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:11.518904 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:11.518905 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:11.519371 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:11.519372 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:11.519662 (+ 290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:11.519663 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:11.519718 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:11.520022 (+ 304us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:48:11.520027 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:48:11.521394 (+ 1367us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.159.190:43653: Server oryx/127.4.159.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":70}
W20260807 08:48:11.521519 18032 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:48:11.518002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.129:59931 (local address 127.4.159.190:43653)
0807 08:48:11.518035 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:11.518036 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:11.518072 (+ 36us) server_negotiation.cc:407] Connection header received
0807 08:48:11.518136 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:11.518137 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:11.518146 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:11.518191 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:11.518417 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:11.518842 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:11.519436 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:11.519609 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:11.519659 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:11.521473 (+ 1814us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.159.129:59931: BlockingRecv error: recv got EOF from 127.4.159.129:59931 (error 108)
Metrics: {"server-negotiator.queue_time_us":13}
W20260807 08:48:11.521727 18024 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 08:48:11.597947 18034 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:48:11.598122 18034 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:48:11.598143 18034 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:48:11.599217 18034 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:48:11.599256 18034 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:48:11.599280 18034 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.159.130
I20260807 08:48:11.600591 18034 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:45469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/krb5kdc/oryx_127.4.159.130.keytab
--principal=oryx/127.4.159.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.159.130:33879
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.159.130
--webserver_port=35907
--tserver_master_addrs=127.4.159.190:43653
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18034
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.159.130
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:48:11.600752 18034 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:48:11.600915 18034 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:48:11.605468 18044 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:48:11.605470 18039 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:48:11.605482 18040 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:48:11.605487 18034 server_base.cc:1034] running on GCE node
I20260807 08:48:11.605847 18034 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:48:11.606030 18034 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:48:11.607162 18034 hybrid_clock.cc:648] HybridClock initialized: now 1786092491607150 us; error 27 us; skew 500 ppm
Aug 07 08:48:11 dist-test-slave-6zbq krb5kdc[16848](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092491, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.159.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:48:11.609422 18034 init.cc:377] Logged in from keytab as oryx/127.4.159.130@KRBTEST.COM (short username oryx)
I20260807 08:48:11.609795 18034 webserver.cc:460] Webserver started at http://127.4.159.130:35907/ using document root <none> and password file <none>
I20260807 08:48:11.609977 18034 fs_manager.cc:359] Metadata directory not provided
I20260807 08:48:11.610038 18034 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:48:11.611249 18034 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:48:11.611726 18055 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:48:11.611868 18034 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 08:48:11.611948 18034 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/ts-1/wal
uuid: "c8e68df9c3e5491b985835084812c821"
format_stamp: "Formatted at 2026-08-07 08:48:10 on dist-test-slave-6zbq"
server_key: "3cbd4b4c44e1e548a761a273c6142175"
server_key_iv: "f4ed6665c513d76a6f6a514cd6df2efa"
server_key_version: "encryptionkey@0"
I20260807 08:48:11.612232 18034 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:48:11.635064 18034 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:48:11.636771 18034 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:48:11.636898 18034 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:48:11.637082 18034 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:48:11.637344 18034 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:48:11.637377 18034 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:48:11.637404 18034 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:48:11.637427 18034 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:48:11.642889 18034 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.130:33879
I20260807 08:48:11.642942 18240 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.130:33879 every 8 connection(s)
I20260807 08:48:11.643226 18034 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/ts-1/data/info.pb
I20260807 08:48:11.644438 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 18034
I20260807 08:48:11.644721 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.159.131:41775
--local_ip_for_outbound_sockets=127.4.159.131
--tserver_master_addrs=127.4.159.190:43653
--webserver_port=42627
--webserver_interface=127.4.159.131
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.131.keytab
--principal=kudu/127.4.159.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:45469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.4.159.131
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/krb5kdc/oryx_127.4.159.131.keytab with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 08:48:11 dist-test-slave-6zbq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.159.130@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:48:11 dist-test-slave-6zbq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.159.130@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM, Server not found in Kerberos database
W20260807 08:48:11.650693 18032 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:48:11.643809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.130:52919 (local address 127.4.159.190:43653)
0807 08:48:11.643865 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:11.643867 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:11.644061 (+ 194us) server_negotiation.cc:407] Connection header received
0807 08:48:11.644516 (+ 455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:11.644518 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:11.644536 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:11.644606 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:11.645524 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:11.646186 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:11.647534 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:11.647722 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:11.647781 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:11.650638 (+ 2857us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.159.130:52919: BlockingRecv error: recv got EOF from 127.4.159.130:52919 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
I20260807 08:48:11.650731 18244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:11.643784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:43653 (local address 127.4.159.130:52919)
0807 08:48:11.643937 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:11.643976 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:11.644420 (+ 444us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:11.644663 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:11.644667 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:11.645259 (+ 592us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:48:11.645433 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:11.645438 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:11.646291 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:11.646292 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:11.647456 (+ 1164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:11.647459 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:11.647805 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:11.647806 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:11.647934 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:11.648381 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:48:11.648391 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:48:11.650552 (+ 2161us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.159.190:43653: Server oryx/127.4.159.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":85,"threads_started":1}
W20260807 08:48:11.650900 18242 heartbeater.cc:644] Failed to heartbeat to 127.4.159.190:43653 (0 consecutive failures): Not authorized: Failed to ping master at 127.4.159.190:43653: Client connection negotiation failed: client connection to 127.4.159.190:43653: Server oryx/127.4.159.190@KRBTEST.COM not found in Kerberos database
Aug 07 08:48:11 dist-test-slave-6zbq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.159.130@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:48:11 dist-test-slave-6zbq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.159.130@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:48:11.655202 18247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:11.651073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:43653 (local address 127.4.159.130:42181)
0807 08:48:11.651183 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:11.651193 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:11.651243 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:11.651363 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:11.651364 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:11.651479 (+ 115us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:48:11.651516 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:11.651517 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:11.652332 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:11.652333 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:11.652842 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:11.652844 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:11.653261 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:11.653262 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:11.653299 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:11.653684 (+ 385us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:48:11.653687 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:48:11.655150 (+ 1463us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.159.190:43653: Server oryx/127.4.159.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":59,"threads_started":1}
W20260807 08:48:11.655314 18032 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:48:11.651118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.130:42181 (local address 127.4.159.190:43653)
0807 08:48:11.651180 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:11.651182 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:11.651247 (+ 65us) server_negotiation.cc:407] Connection header received
0807 08:48:11.651280 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:11.651281 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:11.651292 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:11.651349 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:11.651572 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:11.652244 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:11.652913 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:11.653182 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:11.653253 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:11.655255 (+ 2002us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.159.130:42181: BlockingRecv error: recv got EOF from 127.4.159.130:42181 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Aug 07 08:48:11 dist-test-slave-6zbq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.159.130@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:48:11 dist-test-slave-6zbq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.159.130@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:48:11.659363 18244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:11.655396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:43653 (local address 127.4.159.130:52169)
0807 08:48:11.655458 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:11.655468 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:11.655504 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:11.655615 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:11.655615 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:11.655730 (+ 115us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:48:11.655756 (+ 26us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:11.655757 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:11.656574 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:11.656574 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:11.657038 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:11.657039 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:11.657431 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:11.657432 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:11.657463 (+ 31us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:11.657812 (+ 349us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:48:11.657816 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:48:11.659300 (+ 1484us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.159.190:43653: Server oryx/127.4.159.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":47}
W20260807 08:48:11.659405 18032 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:48:11.655472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.130:52169 (local address 127.4.159.190:43653)
0807 08:48:11.655505 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:11.655506 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:11.655515 (+ 9us) server_negotiation.cc:407] Connection header received
0807 08:48:11.655541 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:11.655542 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:11.655551 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:11.655607 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:11.655812 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:11.656513 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:11.657094 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:11.657377 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:11.657435 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:11.659366 (+ 1931us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.159.130:52169: BlockingRecv error: recv got EOF from 127.4.159.130:52169 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260807 08:48:11.659680 18242 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 08:48:11.726310 18246 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:48:11.726502 18246 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:48:11.726531 18246 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:48:11.727689 18246 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:48:11.727730 18246 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:48:11.727763 18246 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.159.131
I20260807 08:48:11.729214 18246 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:45469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/krb5kdc/oryx_127.4.159.131.keytab
--principal=oryx/127.4.159.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.159.131:41775
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.159.131
--webserver_port=42627
--tserver_master_addrs=127.4.159.190:43653
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18246
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.159.131
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:48:11.729451 18246 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:48:11.729715 18246 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:48:11.735915 18256 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:48:11.738077 18262 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:48:11.738245 18260 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:48:11.738540 18246 server_base.cc:1034] running on GCE node
I20260807 08:48:11.738737 18246 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:48:11.738960 18246 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:48:11.740092 18246 hybrid_clock.cc:648] HybridClock initialized: now 1786092491740078 us; error 31 us; skew 500 ppm
Aug 07 08:48:11 dist-test-slave-6zbq krb5kdc[16848](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092491, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.159.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:48:11.742388 18246 init.cc:377] Logged in from keytab as oryx/127.4.159.131@KRBTEST.COM (short username oryx)
I20260807 08:48:11.742765 18246 webserver.cc:460] Webserver started at http://127.4.159.131:42627/ using document root <none> and password file <none>
I20260807 08:48:11.742967 18246 fs_manager.cc:359] Metadata directory not provided
I20260807 08:48:11.743019 18246 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:48:11.744221 18246 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:48:11.744705 18272 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:48:11.744859 18246 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 08:48:11.744937 18246 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/ts-2/wal
uuid: "69851855ca6d4c6680181bf785c8b02c"
format_stamp: "Formatted at 2026-08-07 08:48:11 on dist-test-slave-6zbq"
server_key: "794737083233a9af74d55b23a9268273"
server_key_iv: "17d189d32b1e8bb2d5fa3cd7f4ec2cca"
server_key_version: "encryptionkey@0"
I20260807 08:48:11.745227 18246 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:48:11.754618 18246 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:48:11.756479 18246 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:48:11.756588 18246 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:48:11.756774 18246 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:48:11.757031 18246 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:48:11.757063 18246 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:48:11.757082 18246 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:48:11.757104 18246 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:48:11.762454 18246 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.131:41775
I20260807 08:48:11.762503 18439 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.131:41775 every 8 connection(s)
I20260807 08:48:11.762781 18246 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786092453486433-4734-0/minicluster-data/ts-2/data/info.pb
Aug 07 08:48:11 dist-test-slave-6zbq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.159.131@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:48:11 dist-test-slave-6zbq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.159.131@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM, Server not found in Kerberos database
W20260807 08:48:11.768947 18032 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:48:11.763386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.131:40183 (local address 127.4.159.190:43653)
0807 08:48:11.763446 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:11.763448 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:11.763596 (+ 148us) server_negotiation.cc:407] Connection header received
0807 08:48:11.764070 (+ 474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:11.764071 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:11.764079 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:11.764136 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:11.764925 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:11.765395 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:11.766257 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:11.766475 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:11.766519 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:11.768892 (+ 2373us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.159.131:40183: BlockingRecv error: recv got EOF from 127.4.159.131:40183 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
I20260807 08:48:11.768975 18445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:11.763356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:43653 (local address 127.4.159.131:40183)
0807 08:48:11.763496 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:11.763533 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:11.763962 (+ 429us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:11.764177 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:11.764180 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:11.764699 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:48:11.764864 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:11.764869 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:11.765455 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:11.765455 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:11.766191 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:11.766193 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:11.766538 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:11.766539 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:11.766634 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:11.766967 (+ 333us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:48:11.766975 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:48:11.768805 (+ 1830us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.159.190:43653: Server oryx/127.4.159.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":79,"threads_started":1}
W20260807 08:48:11.769238 18440 heartbeater.cc:644] Failed to heartbeat to 127.4.159.190:43653 (0 consecutive failures): Not authorized: Failed to ping master at 127.4.159.190:43653: Client connection negotiation failed: client connection to 127.4.159.190:43653: Server oryx/127.4.159.190@KRBTEST.COM not found in Kerberos database
I20260807 08:48:11.770191 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 18246
Aug 07 08:48:11 dist-test-slave-6zbq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.159.131@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:48:11 dist-test-slave-6zbq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.159.131@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:48:11.773555 18445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:11.769434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:43653 (local address 127.4.159.131:37129)
0807 08:48:11.769476 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:11.769484 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:11.769513 (+ 29us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:11.769697 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:11.769698 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:11.769812 (+ 114us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:48:11.769852 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:11.769853 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:11.770493 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:11.770494 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:11.771212 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:11.771215 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:11.771501 (+ 286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:11.771502 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:11.771564 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:11.771915 (+ 351us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:48:11.771918 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:48:11.773494 (+ 1576us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.159.190:43653: Server oryx/127.4.159.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":26}
W20260807 08:48:11.773617 18032 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:48:11.769561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.131:37129 (local address 127.4.159.190:43653)
0807 08:48:11.769605 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:11.769607 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:11.769613 (+ 6us) server_negotiation.cc:407] Connection header received
0807 08:48:11.769631 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:11.769631 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:11.769640 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:11.769682 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:11.769915 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:11.770426 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:11.771276 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:11.771443 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:11.771484 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:11.773563 (+ 2079us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.159.131:37129: BlockingRecv error: recv got EOF from 127.4.159.131:37129 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
Aug 07 08:48:11 dist-test-slave-6zbq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.159.131@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:48:11 dist-test-slave-6zbq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.159.131@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:48:11.777671 18448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:11.770547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33028 (local address 127.4.159.190:43653)
0807 08:48:11.770655 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:11.770657 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:11.770812 (+ 155us) server_negotiation.cc:407] Connection header received
0807 08:48:11.770876 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:11.770876 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:11.770885 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:11.770933 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:11.771556 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:11.772087 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:11.772878 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:11.773193 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:11.773260 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:11.774506 (+ 1246us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:48:11.774511 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:48:11.774511 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:48:11.774522 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:48:11.776070 (+ 1548us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:48:11.776447 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:48:11.776450 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:48:11.776451 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:48:11.776564 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:48:11.776832 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:48:11.776833 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:48:11.776834 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:48:11.777139 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:48:11.777277 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0807 08:48:11.777479 (+ 202us) server_negotiation.cc:299] Negotiation successful
0807 08:48:11.777586 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91,"thread_start_us":63,"threads_started":1}
I20260807 08:48:11.777927 18445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:11.773869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:43653 (local address 127.4.159.131:51683)
0807 08:48:11.773907 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:11.773916 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:11.773944 (+ 28us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:11.774061 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:11.774061 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:11.774187 (+ 126us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:48:11.774210 (+ 23us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:11.774212 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:11.774960 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:11.774962 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:11.775514 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:11.775517 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:11.775932 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:11.775932 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:11.775964 (+ 32us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:11.776249 (+ 285us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:48:11.776253 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:48:11.777868 (+ 1615us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.159.190:43653: Server oryx/127.4.159.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":25}
W20260807 08:48:11.777978 18032 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:48:11.773926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.131:51683 (local address 127.4.159.190:43653)
0807 08:48:11.773980 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:11.773981 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:11.773989 (+ 8us) server_negotiation.cc:407] Connection header received
0807 08:48:11.774006 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:11.774007 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:11.774015 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:11.774058 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:11.774273 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:11.774884 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:11.775588 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:11.775861 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:11.775926 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:11.777935 (+ 2009us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.159.131:51683: BlockingRecv error: recv got EOF from 127.4.159.131:51683 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
W20260807 08:48:11.778129 18440 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 07 08:48:12 dist-test-slave-6zbq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.159.129@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:48:12 dist-test-slave-6zbq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.159.129@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:48:12.526373 18450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:12.522046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:43653 (local address 127.4.159.129:43997)
0807 08:48:12.522155 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:12.522165 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:12.522219 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:12.522408 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:12.522409 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:12.522582 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:48:12.522634 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:12.522635 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:12.523437 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:12.523438 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:12.523971 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:12.523973 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:12.524272 (+ 299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:12.524273 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:12.524310 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:12.524725 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:48:12.524731 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:48:12.526283 (+ 1552us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.159.190:43653: Server oryx/127.4.159.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":63,"threads_started":1}
W20260807 08:48:12.526454 18451 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:48:12.522136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.129:43997 (local address 127.4.159.190:43653)
0807 08:48:12.522266 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:12.522268 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:12.522276 (+ 8us) server_negotiation.cc:407] Connection header received
0807 08:48:12.522298 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:12.522298 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:12.522308 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:12.522382 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:12.522691 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:12.523362 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:12.524030 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:12.524224 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:12.524268 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:12.526395 (+ 2127us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.159.129:43997: BlockingRecv error: recv got EOF from 127.4.159.129:43997 (error 108)
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":73,"threads_started":1}
Aug 07 08:48:12 dist-test-slave-6zbq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.159.130@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:48:12 dist-test-slave-6zbq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.159.130@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:48:12.664521 18453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:12.660007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:43653 (local address 127.4.159.130:49845)
0807 08:48:12.660126 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:12.660137 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:12.660204 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:12.660317 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:12.660318 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:12.660469 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:48:12.660516 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:12.660517 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:12.661301 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:12.661302 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:12.661804 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:12.661806 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:12.662136 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:12.662137 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:12.662201 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:12.662641 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:48:12.662645 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:48:12.664455 (+ 1810us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.159.190:43653: Server oryx/127.4.159.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":56,"threads_started":1}
W20260807 08:48:12.664716 18451 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:48:12.660064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.130:49845 (local address 127.4.159.190:43653)
0807 08:48:12.660143 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:12.660145 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:12.660189 (+ 44us) server_negotiation.cc:407] Connection header received
0807 08:48:12.660251 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:12.660252 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:12.660261 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:12.660306 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:12.660590 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:12.661229 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:12.661880 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:12.662077 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:12.662131 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:12.664663 (+ 2532us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.159.130:49845: BlockingRecv error: recv got EOF from 127.4.159.130:49845 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 07 08:48:12 dist-test-slave-6zbq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.159.131@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:48:12 dist-test-slave-6zbq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.159.131@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:48:12.782547 18454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:12.778435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:43653 (local address 127.4.159.131:39455)
0807 08:48:12.778544 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:12.778555 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:12.778618 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:12.778733 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:12.778733 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:12.778896 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:48:12.778939 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:12.778941 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:12.779559 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:12.779560 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:12.780088 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:12.780090 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:12.780412 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:12.780413 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:12.780463 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:12.780841 (+ 378us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:48:12.780845 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:48:12.782488 (+ 1643us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.159.190:43653: Server oryx/127.4.159.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":86,"thread_start_us":57,"threads_started":1}
W20260807 08:48:12.782655 18451 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:48:12.778499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.131:39455 (local address 127.4.159.190:43653)
0807 08:48:12.778553 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:12.778556 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:12.778599 (+ 43us) server_negotiation.cc:407] Connection header received
0807 08:48:12.778673 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:12.778674 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:12.778686 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:12.778729 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:12.779004 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:12.779491 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:12.780161 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:12.780356 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:12.780409 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:12.782586 (+ 2177us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.159.131:39455: BlockingRecv error: recv got EOF from 127.4.159.131:39455 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
Aug 07 08:48:13 dist-test-slave-6zbq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.159.129@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:48:13 dist-test-slave-6zbq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.159.129@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:48:13.531088 18455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:13.526798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:43653 (local address 127.4.159.129:32897)
0807 08:48:13.526915 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:13.526925 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:13.526975 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:13.527102 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:13.527102 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:13.527266 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:48:13.527316 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:13.527317 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:13.527914 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:13.527914 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:13.528432 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:13.528433 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:13.528749 (+ 316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:13.528750 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:13.528794 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:13.529215 (+ 421us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:48:13.529219 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:48:13.531033 (+ 1814us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.159.190:43653: Server oryx/127.4.159.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":62,"threads_started":1}
W20260807 08:48:13.531178 18456 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:48:13.526871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.129:32897 (local address 127.4.159.190:43653)
0807 08:48:13.526990 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:13.526992 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:13.527002 (+ 10us) server_negotiation.cc:407] Connection header received
0807 08:48:13.527024 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:13.527024 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:13.527034 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:13.527092 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:13.527368 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:13.527862 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:13.528494 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:13.528692 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:13.528745 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:13.531113 (+ 2368us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.159.129:32897: BlockingRecv error: recv got EOF from 127.4.159.129:32897 (error 108)
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":62,"threads_started":1}
Aug 07 08:48:13 dist-test-slave-6zbq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.159.130@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:48:13 dist-test-slave-6zbq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.159.130@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:48:13.669294 18457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:13.664918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:43653 (local address 127.4.159.130:37665)
0807 08:48:13.665067 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:13.665077 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:13.665146 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:13.665257 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:13.665258 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:13.665414 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:48:13.665464 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:13.665465 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:13.666059 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:13.666059 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:13.666648 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:13.666651 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:13.666947 (+ 296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:13.666947 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:13.666988 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:13.667404 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:48:13.667408 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:48:13.669222 (+ 1814us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.159.190:43653: Server oryx/127.4.159.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":88,"threads_started":1}
W20260807 08:48:13.669373 18456 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:48:13.665001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.130:37665 (local address 127.4.159.190:43653)
0807 08:48:13.665056 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:13.665058 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:13.665128 (+ 70us) server_negotiation.cc:407] Connection header received
0807 08:48:13.665194 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:13.665194 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:13.665205 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:13.665251 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:13.665509 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:13.666007 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:13.666712 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:13.666896 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:13.666939 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:13.669321 (+ 2382us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.159.130:37665: BlockingRecv error: recv got EOF from 127.4.159.130:37665 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
Aug 07 08:48:13 dist-test-slave-6zbq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.159.131@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:48:13 dist-test-slave-6zbq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.159.131@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:48:13.787209 18458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:13.782951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:43653 (local address 127.4.159.131:60065)
0807 08:48:13.783087 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:13.783096 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:13.783135 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:13.783279 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:13.783280 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:13.783429 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:48:13.783475 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:13.783477 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:13.784077 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:13.784078 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:13.784591 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:13.784593 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:13.785002 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:13.785004 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:13.785070 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:13.785454 (+ 384us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:48:13.785458 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:48:13.787139 (+ 1681us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.159.190:43653: Server oryx/127.4.159.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":78,"threads_started":1}
W20260807 08:48:13.787279 18456 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:48:13.783073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.131:60065 (local address 127.4.159.190:43653)
0807 08:48:13.783172 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:13.783174 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:13.783182 (+ 8us) server_negotiation.cc:407] Connection header received
0807 08:48:13.783204 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:13.783205 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:13.783216 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:13.783265 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:13.783531 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:13.784023 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:13.784661 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:13.784932 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:13.784992 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:13.787226 (+ 2234us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.159.131:60065: BlockingRecv error: recv got EOF from 127.4.159.131:60065 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 07 08:48:14 dist-test-slave-6zbq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.159.129@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:48:14 dist-test-slave-6zbq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.159.129@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:48:14.535759 18459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:14.531511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:43653 (local address 127.4.159.129:45297)
0807 08:48:14.531643 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:14.531654 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:14.531703 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:14.531814 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:14.531815 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:14.531969 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:48:14.532018 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:14.532019 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:14.532678 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:14.532678 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:14.533202 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:14.533203 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:14.533516 (+ 313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:14.533516 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:14.533551 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:14.533937 (+ 386us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:48:14.533941 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:48:14.535671 (+ 1730us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.159.190:43653: Server oryx/127.4.159.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":68,"threads_started":1}
W20260807 08:48:14.535827 18461 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:48:14.531596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.129:45297 (local address 127.4.159.190:43653)
0807 08:48:14.531702 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:14.531705 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:14.531713 (+ 8us) server_negotiation.cc:407] Connection header received
0807 08:48:14.531749 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:14.531749 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:14.531759 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:14.531804 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:14.532093 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:14.532616 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:14.533271 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:14.533464 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:14.533514 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:14.535766 (+ 2252us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.159.129:45297: BlockingRecv error: recv got EOF from 127.4.159.129:45297 (error 108)
Metrics: {"server-negotiator.queue_time_us":87,"thread_start_us":51,"threads_started":1}
Aug 07 08:48:14 dist-test-slave-6zbq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.159.130@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:48:14 dist-test-slave-6zbq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.159.130@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:48:14.673861 18464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:14.669707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:43653 (local address 127.4.159.130:51255)
0807 08:48:14.669827 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:14.669837 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:14.669890 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:14.670004 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:14.670005 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:14.670159 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:48:14.670220 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:14.670222 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:14.670828 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:14.670829 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:14.671349 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:14.671351 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:14.671663 (+ 312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:14.671664 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:14.671699 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:14.672068 (+ 369us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:48:14.672073 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:48:14.673804 (+ 1731us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.159.190:43653: Server oryx/127.4.159.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":54,"threads_started":1}
W20260807 08:48:14.673949 18461 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:48:14.669765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.130:51255 (local address 127.4.159.190:43653)
0807 08:48:14.669818 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:14.669820 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:14.669884 (+ 64us) server_negotiation.cc:407] Connection header received
0807 08:48:14.669930 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:14.669931 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:14.669942 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:14.669995 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:14.670267 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:14.670777 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:14.671421 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:14.671611 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:14.671662 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:14.673892 (+ 2230us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.159.130:51255: BlockingRecv error: recv got EOF from 127.4.159.130:51255 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
Aug 07 08:48:14 dist-test-slave-6zbq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.159.131@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:48:14 dist-test-slave-6zbq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.159.131@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:48:14.791745 18465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:14.787592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:43653 (local address 127.4.159.131:33981)
0807 08:48:14.787698 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:14.787709 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:14.787763 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:14.787871 (+ 108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:14.787871 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:14.788014 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:48:14.788061 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:14.788063 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:14.788662 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:14.788663 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:14.789331 (+ 668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:14.789334 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:14.789629 (+ 295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:14.789630 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:14.789666 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:14.790023 (+ 357us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:48:14.790027 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:48:14.791659 (+ 1632us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.159.190:43653: Server oryx/127.4.159.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":86,"thread_start_us":52,"threads_started":1}
W20260807 08:48:14.791801 18461 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:48:14.787657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.131:33981 (local address 127.4.159.190:43653)
0807 08:48:14.787721 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:14.787723 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:14.787748 (+ 25us) server_negotiation.cc:407] Connection header received
0807 08:48:14.787809 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:14.787810 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:14.787821 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:14.787865 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:14.788116 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:14.788591 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:14.789395 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:14.789577 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:14.789627 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:14.791749 (+ 2122us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.159.131:33981: BlockingRecv error: recv got EOF from 127.4.159.131:33981 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 07 08:48:15 dist-test-slave-6zbq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.159.129@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:48:15 dist-test-slave-6zbq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.159.129@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:48:15.540382 18466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:15.536221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:43653 (local address 127.4.159.129:50107)
0807 08:48:15.536377 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:15.536390 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:15.536442 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:15.536568 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:15.536569 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:15.536724 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:48:15.536774 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:15.536775 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:15.537373 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:15.537374 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:15.537901 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:15.537903 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:15.538172 (+ 269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:15.538172 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:15.538206 (+ 34us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:15.538621 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:48:15.538625 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:48:15.540323 (+ 1698us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.159.190:43653: Server oryx/127.4.159.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":64,"threads_started":1}
W20260807 08:48:15.540477 18467 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:48:15.536286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.129:50107 (local address 127.4.159.190:43653)
0807 08:48:15.536412 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:15.536414 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:15.536426 (+ 12us) server_negotiation.cc:407] Connection header received
0807 08:48:15.536491 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:15.536492 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:15.536504 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:15.536558 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:15.536833 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:15.537323 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:15.537951 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:15.538129 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:15.538170 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:15.540411 (+ 2241us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.159.129:50107: BlockingRecv error: recv got EOF from 127.4.159.129:50107 (error 108)
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":69,"threads_started":1}
Aug 07 08:48:15 dist-test-slave-6zbq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.159.130@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:48:15 dist-test-slave-6zbq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.159.130@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:48:15.682430 18469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:15.674315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:43653 (local address 127.4.159.130:35023)
0807 08:48:15.674457 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:15.674470 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:15.674543 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:15.674675 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:15.674677 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:15.674848 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:48:15.674895 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:15.674897 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:15.675518 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:15.675519 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:15.676025 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:15.676027 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:15.676348 (+ 321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:15.676349 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:15.676406 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:15.680638 (+ 4232us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:48:15.680643 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:48:15.682366 (+ 1723us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.159.190:43653: Server oryx/127.4.159.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":88,"threads_started":1}
W20260807 08:48:15.682508 18467 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:48:15.674431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.130:35023 (local address 127.4.159.190:43653)
0807 08:48:15.674496 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:15.674498 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:15.674523 (+ 25us) server_negotiation.cc:407] Connection header received
0807 08:48:15.674604 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:15.674606 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:15.674618 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:15.674666 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:15.674959 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:15.675448 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:15.676095 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:15.676289 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:15.676345 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:15.682449 (+ 6104us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.159.130:35023: BlockingRecv error: recv got EOF from 127.4.159.130:35023 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 07 08:48:15 dist-test-slave-6zbq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.159.131@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:48:15 dist-test-slave-6zbq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.159.131@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:48:15.796758 18470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:15.792146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:43653 (local address 127.4.159.131:46371)
0807 08:48:15.792294 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:15.792305 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:15.792367 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:15.792498 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:15.792498 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:15.792704 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:48:15.792763 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:15.792766 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:15.793497 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:15.793498 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:15.794131 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:15.794134 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:15.794522 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:15.794523 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:15.794594 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:15.795068 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:48:15.795074 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:48:15.796701 (+ 1627us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.159.190:43653: Server oryx/127.4.159.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":66,"threads_started":1}
W20260807 08:48:15.796844 18467 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:48:15.792203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.131:46371 (local address 127.4.159.190:43653)
0807 08:48:15.792278 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:15.792280 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:15.792359 (+ 79us) server_negotiation.cc:407] Connection header received
0807 08:48:15.792423 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:15.792424 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:15.792438 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:15.792481 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:15.792835 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:15.793427 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:15.794208 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:15.794464 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:15.794522 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:15.796785 (+ 2263us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.159.131:46371: BlockingRecv error: recv got EOF from 127.4.159.131:46371 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 07 08:48:16 dist-test-slave-6zbq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.159.129@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:48:16 dist-test-slave-6zbq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.159.129@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:48:16.545240 18472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:16.540781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:43653 (local address 127.4.159.129:49253)
0807 08:48:16.540913 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:16.540924 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:16.540969 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:16.541124 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:16.541125 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:16.541279 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:48:16.541329 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:16.541330 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:16.542026 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:16.542027 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:16.542601 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:16.542604 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:16.542918 (+ 314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:16.542919 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:16.542956 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:16.543355 (+ 399us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:48:16.543359 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:48:16.545180 (+ 1821us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.159.190:43653: Server oryx/127.4.159.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":59,"threads_started":1}
W20260807 08:48:16.545361 18474 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:48:16.540844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.129:49253 (local address 127.4.159.190:43653)
0807 08:48:16.540962 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:16.540964 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:16.540975 (+ 11us) server_negotiation.cc:407] Connection header received
0807 08:48:16.541014 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:16.541014 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:16.541024 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:16.541114 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:16.541387 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:16.541966 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:16.542669 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:16.542862 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:16.542911 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:16.545284 (+ 2373us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.159.129:49253: BlockingRecv error: recv got EOF from 127.4.159.129:49253 (error 108)
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":57,"threads_started":1}
Aug 07 08:48:16 dist-test-slave-6zbq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.159.130@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:48:16 dist-test-slave-6zbq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.159.130@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:48:16.687040 18475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:16.682830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:43653 (local address 127.4.159.130:41161)
0807 08:48:16.683000 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:16.683012 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:16.683070 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:16.683178 (+ 108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:16.683179 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:16.683336 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:48:16.683384 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:16.683386 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:16.683978 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:16.683978 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:16.684498 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:16.684499 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:16.684804 (+ 305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:16.684804 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:16.684841 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:16.685269 (+ 428us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:48:16.685274 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:48:16.686982 (+ 1708us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.159.190:43653: Server oryx/127.4.159.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":87,"threads_started":1}
W20260807 08:48:16.687148 18474 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:48:16.682953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.130:41161 (local address 127.4.159.190:43653)
0807 08:48:16.683016 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:16.683018 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:16.683052 (+ 34us) server_negotiation.cc:407] Connection header received
0807 08:48:16.683114 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:16.683115 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:16.683125 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:16.683168 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:16.683434 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:16.683923 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:16.684559 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:16.684747 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:16.684795 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:16.687086 (+ 2291us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.159.130:41161: BlockingRecv error: recv got EOF from 127.4.159.130:41161 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Aug 07 08:48:16 dist-test-slave-6zbq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.159.131@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:48:16 dist-test-slave-6zbq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.159.131@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:48:16.801342 18477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:16.797162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:43653 (local address 127.4.159.131:36319)
0807 08:48:16.797328 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:16.797338 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:16.797395 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:16.797513 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:16.797514 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:16.797672 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:48:16.797721 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:16.797723 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:16.798312 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:16.798312 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:16.798839 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:16.798840 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:16.799151 (+ 311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:16.799152 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:16.799191 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:16.799599 (+ 408us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:48:16.799603 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:48:16.801284 (+ 1681us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.159.190:43653: Server oryx/127.4.159.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":103,"threads_started":1}
W20260807 08:48:16.801442 18474 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:48:16.797237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.131:36319 (local address 127.4.159.190:43653)
0807 08:48:16.797297 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:16.797300 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:16.797392 (+ 92us) server_negotiation.cc:407] Connection header received
0807 08:48:16.797446 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:16.797447 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:16.797460 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:16.797497 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:16.797774 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:16.798258 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:16.798897 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:16.799075 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:16.799123 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:16.801379 (+ 2256us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.159.131:36319: BlockingRecv error: recv got EOF from 127.4.159.131:36319 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Aug 07 08:48:17 dist-test-slave-6zbq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.159.129@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:48:17 dist-test-slave-6zbq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.159.129@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:48:17.550519 18478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:17.545689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:43653 (local address 127.4.159.129:57365)
0807 08:48:17.545861 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:17.545875 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:17.545934 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:17.546060 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:17.546060 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:17.546265 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:48:17.546329 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:17.546331 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:17.547029 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:17.547031 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:17.547612 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:17.547615 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:17.547973 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:17.547974 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:17.548041 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:17.548464 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:48:17.548470 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:48:17.550440 (+ 1970us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.159.190:43653: Server oryx/127.4.159.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":85,"threads_started":1}
W20260807 08:48:17.550609 18479 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:48:17.545789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.129:57365 (local address 127.4.159.190:43653)
0807 08:48:17.545924 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:17.545925 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:17.545933 (+ 8us) server_negotiation.cc:407] Connection header received
0807 08:48:17.545980 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:17.545980 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:17.545990 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:17.546046 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:17.546441 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:17.546945 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:17.547702 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:17.547900 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:17.547956 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:17.550546 (+ 2590us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.159.129:57365: BlockingRecv error: recv got EOF from 127.4.159.129:57365 (error 108)
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":72,"threads_started":1}
Aug 07 08:48:17 dist-test-slave-6zbq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.159.130@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:48:17 dist-test-slave-6zbq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.159.130@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:48:17.691854 18480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:17.687466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:43653 (local address 127.4.159.130:56063)
0807 08:48:17.687596 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:17.687617 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:17.687691 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:17.687843 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:17.687844 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:17.688005 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:48:17.688051 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:17.688053 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:17.688712 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:17.688714 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:17.689219 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:17.689220 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:17.689567 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:17.689568 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:17.689630 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:17.690058 (+ 428us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:48:17.690062 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:48:17.691796 (+ 1734us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.159.190:43653: Server oryx/127.4.159.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":64,"threads_started":1}
W20260807 08:48:17.691969 18479 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:48:17.687551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.130:56063 (local address 127.4.159.190:43653)
0807 08:48:17.687619 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:17.687621 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:17.687686 (+ 65us) server_negotiation.cc:407] Connection header received
0807 08:48:17.687766 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:17.687767 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:17.687780 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:17.687825 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:17.688147 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:17.688634 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:17.689306 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:17.689497 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:17.689552 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:17.691896 (+ 2344us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.159.130:56063: BlockingRecv error: recv got EOF from 127.4.159.130:56063 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 07 08:48:17 dist-test-slave-6zbq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.159.131@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:48:17 dist-test-slave-6zbq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.159.131@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:48:17.806114 18481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:17.801750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:43653 (local address 127.4.159.131:41331)
0807 08:48:17.801876 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:17.801888 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:17.801946 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:17.802064 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:17.802064 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:17.802226 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:48:17.802276 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:17.802278 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:17.802909 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:17.802910 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:17.803580 (+ 670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:17.803583 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:17.803977 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:17.803978 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:17.804019 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:17.804443 (+ 424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:48:17.804448 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:48:17.806034 (+ 1586us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.159.190:43653: Server oryx/127.4.159.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":50,"threads_started":1}
W20260807 08:48:17.806376 18479 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:48:17.801816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.131:41331 (local address 127.4.159.190:43653)
0807 08:48:17.801891 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:17.801893 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:17.801942 (+ 49us) server_negotiation.cc:407] Connection header received
0807 08:48:17.801997 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:17.801998 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:17.802010 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:17.802047 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:17.802331 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:17.802851 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:17.803665 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:17.803911 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:17.803962 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:17.806160 (+ 2198us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.159.131:41331: BlockingRecv error: recv got EOF from 127.4.159.131:41331 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 07 08:48:18 dist-test-slave-6zbq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.159.129@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:48:18 dist-test-slave-6zbq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.159.129@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:48:18.555413 18482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:18.551003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:43653 (local address 127.4.159.129:59003)
0807 08:48:18.551158 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:18.551171 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:18.551235 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:18.551377 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:18.551378 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:18.551548 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:48:18.551597 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:18.551599 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:18.552245 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:18.552246 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:18.552770 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:18.552772 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:18.553128 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:18.553129 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:18.553187 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:18.553615 (+ 428us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:48:18.553620 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:48:18.555353 (+ 1733us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.159.190:43653: Server oryx/127.4.159.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":63,"threads_started":1}
W20260807 08:48:18.555546 18483 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:48:18.551039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.129:59003 (local address 127.4.159.190:43653)
0807 08:48:18.551161 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:18.551163 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:18.551216 (+ 53us) server_negotiation.cc:407] Connection header received
0807 08:48:18.551282 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:18.551283 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:18.551297 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:18.551355 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:18.551674 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:18.552164 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:18.552851 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:18.553049 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:18.553103 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:18.555476 (+ 2373us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.159.129:59003: BlockingRecv error: recv got EOF from 127.4.159.129:59003 (error 108)
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":63,"threads_started":1}
Aug 07 08:48:18 dist-test-slave-6zbq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.159.130@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:48:18 dist-test-slave-6zbq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.159.130@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:48:18.696610 18484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:18.692349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:43653 (local address 127.4.159.130:36845)
0807 08:48:18.692489 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:18.692501 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:18.692560 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:18.692695 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:18.692696 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:18.692853 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:48:18.692901 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:18.692903 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:18.693522 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:18.693523 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:18.694041 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:18.694043 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:18.694386 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:18.694387 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:18.694425 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:18.694822 (+ 397us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:48:18.694827 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:48:18.696550 (+ 1723us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.159.190:43653: Server oryx/127.4.159.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":62,"threads_started":1}
W20260807 08:48:18.696684 18483 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:48:18.692446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.130:36845 (local address 127.4.159.190:43653)
0807 08:48:18.692555 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:18.692557 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:18.692570 (+ 13us) server_negotiation.cc:407] Connection header received
0807 08:48:18.692621 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:18.692622 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:18.692634 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:18.692683 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:18.692956 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:18.693449 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:18.694102 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:18.694308 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:18.694381 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:18.696636 (+ 2255us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.159.130:36845: BlockingRecv error: recv got EOF from 127.4.159.130:36845 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Aug 07 08:48:18 dist-test-slave-6zbq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.159.131@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:48:18 dist-test-slave-6zbq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.159.131@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:48:18.811002 18488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:18.806590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:43653 (local address 127.4.159.131:56405)
0807 08:48:18.806736 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:18.806746 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:18.806800 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:18.806910 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:18.806911 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:18.807092 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:48:18.807144 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:18.807147 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:18.807870 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:18.807872 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:18.808534 (+ 662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:18.808538 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:18.808868 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:18.808869 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:18.808910 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:18.809312 (+ 402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:48:18.809316 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:48:18.810946 (+ 1630us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.159.190:43653: Server oryx/127.4.159.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":77,"threads_started":1}
W20260807 08:48:18.811092 18483 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:48:18.806644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.131:56405 (local address 127.4.159.190:43653)
0807 08:48:18.806697 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:18.806698 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:18.806797 (+ 99us) server_negotiation.cc:407] Connection header received
0807 08:48:18.806838 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:18.806839 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:18.806850 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:18.806894 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:18.807219 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:18.807791 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:18.808617 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:18.808803 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:18.808855 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:18.811033 (+ 2178us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.159.131:56405: BlockingRecv error: recv got EOF from 127.4.159.131:56405 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
Aug 07 08:48:19 dist-test-slave-6zbq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.159.129@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:48:19 dist-test-slave-6zbq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.159.129@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:48:19.560266 18489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:19.555871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:43653 (local address 127.4.159.129:55185)
0807 08:48:19.556020 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:19.556030 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:19.556076 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:19.556236 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:19.556237 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:19.556394 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:48:19.556444 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:19.556445 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:19.557066 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:19.557067 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:19.557593 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:19.557595 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:19.557887 (+ 292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:19.557888 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:19.557925 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:19.558372 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:48:19.558385 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:48:19.560204 (+ 1819us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.159.190:43653: Server oryx/127.4.159.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":67,"threads_started":1}
W20260807 08:48:19.560436 18491 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:48:19.555989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.129:55185 (local address 127.4.159.190:43653)
0807 08:48:19.556125 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:19.556126 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:19.556134 (+ 8us) server_negotiation.cc:407] Connection header received
0807 08:48:19.556155 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:19.556155 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:19.556166 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:19.556222 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:19.556502 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:19.557005 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:19.557655 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:19.557835 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:19.557878 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:19.560352 (+ 2474us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.159.129:55185: BlockingRecv error: recv got EOF from 127.4.159.129:55185 (error 108)
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":56,"threads_started":1}
Aug 07 08:48:19 dist-test-slave-6zbq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.159.130@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:48:19 dist-test-slave-6zbq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.159.130@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:48:19.701668 18492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:19.697089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:43653 (local address 127.4.159.130:45217)
0807 08:48:19.697250 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:19.697264 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:19.697339 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:19.697495 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:19.697496 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:19.697673 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:48:19.697719 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:19.697721 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:19.698562 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:19.698563 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:19.699071 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:19.699072 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:19.699426 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:19.699427 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:19.699483 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:19.699874 (+ 391us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:48:19.699878 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:48:19.701593 (+ 1715us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.159.190:43653: Server oryx/127.4.159.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":80,"threads_started":1}
W20260807 08:48:19.701763 18491 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:48:19.697177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.130:45217 (local address 127.4.159.190:43653)
0807 08:48:19.697273 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:19.697274 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:19.697317 (+ 43us) server_negotiation.cc:407] Connection header received
0807 08:48:19.697416 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:19.697416 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:19.697429 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:19.697476 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:19.697798 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:19.698486 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:19.699155 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:19.699357 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:19.699413 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:19.701705 (+ 2292us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.159.130:45217: BlockingRecv error: recv got EOF from 127.4.159.130:45217 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 07 08:48:19 dist-test-slave-6zbq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.159.131@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:48:19 dist-test-slave-6zbq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.159.131@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:48:19.815876 18493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:19.811447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:43653 (local address 127.4.159.131:34341)
0807 08:48:19.811611 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:19.811622 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:19.811679 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:19.811803 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:19.811804 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:19.811956 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:48:19.812004 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:19.812006 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:19.812592 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:19.812593 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:19.813144 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:19.813146 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:19.813432 (+ 286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:19.813432 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:19.813471 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:19.813880 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:48:19.813884 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:48:19.815782 (+ 1898us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.159.190:43653: Server oryx/127.4.159.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":86,"threads_started":1}
W20260807 08:48:19.815936 18491 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:48:19.811523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.131:34341 (local address 127.4.159.190:43653)
0807 08:48:19.811573 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:19.811574 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:19.811673 (+ 99us) server_negotiation.cc:407] Connection header received
0807 08:48:19.811734 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:19.811735 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:19.811747 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:19.811785 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:19.812053 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:19.812534 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:19.813204 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:19.813379 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:19.813422 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:19.815884 (+ 2462us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.159.131:34341: BlockingRecv error: recv got EOF from 127.4.159.131:34341 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
Aug 07 08:48:20 dist-test-slave-6zbq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.159.129@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:48:20 dist-test-slave-6zbq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.159.129@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:48:20.565059 18494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:20.560750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:43653 (local address 127.4.159.129:36795)
0807 08:48:20.560891 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:20.560902 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:20.560950 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:20.561106 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:20.561107 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:20.561275 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:48:20.561325 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:20.561327 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:20.561940 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:20.561940 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:20.562497 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:20.562499 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:20.562805 (+ 306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:20.562806 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:20.562844 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:20.563265 (+ 421us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:48:20.563270 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:48:20.565001 (+ 1731us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.159.190:43653: Server oryx/127.4.159.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":56,"threads_started":1}
W20260807 08:48:20.565177 18495 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:48:20.560810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.129:36795 (local address 127.4.159.190:43653)
0807 08:48:20.560941 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:20.560943 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:20.560952 (+ 9us) server_negotiation.cc:407] Connection header received
0807 08:48:20.560994 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:20.560994 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:20.561006 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:20.561067 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:20.561383 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:20.561882 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:20.562555 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:20.562750 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:20.562796 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:20.565107 (+ 2311us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.159.129:36795: BlockingRecv error: recv got EOF from 127.4.159.129:36795 (error 108)
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":72,"threads_started":1}
Aug 07 08:48:20 dist-test-slave-6zbq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.159.130@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:48:20 dist-test-slave-6zbq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.159.130@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:48:20.710645 18496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:20.702151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:43653 (local address 127.4.159.130:59143)
0807 08:48:20.702313 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:20.702323 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:20.702396 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:20.702514 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:20.702515 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:20.702678 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:48:20.702724 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:20.702726 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:20.703317 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:20.703317 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:20.703819 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:20.703820 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:20.704141 (+ 321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:20.704142 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:20.704179 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:20.708757 (+ 4578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:48:20.708762 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:48:20.710585 (+ 1823us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.159.190:43653: Server oryx/127.4.159.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":93,"threads_started":1}
W20260807 08:48:20.710736 18495 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:48:20.702251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.130:59143 (local address 127.4.159.190:43653)
0807 08:48:20.702314 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:20.702315 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:20.702395 (+ 80us) server_negotiation.cc:407] Connection header received
0807 08:48:20.702439 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:20.702440 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:20.702451 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:20.702502 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:20.702775 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:20.703263 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:20.703884 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:20.704073 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:20.704133 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:20.710674 (+ 6541us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.159.130:59143: BlockingRecv error: recv got EOF from 127.4.159.130:59143 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Aug 07 08:48:20 dist-test-slave-6zbq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.159.131@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:48:20 dist-test-slave-6zbq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.159.131@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:48:20.820667 18497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:20.816356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:43653 (local address 127.4.159.131:36719)
0807 08:48:20.816482 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:20.816495 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:20.816555 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:20.816677 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:20.816677 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:20.816835 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:48:20.816883 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:20.816885 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:20.817493 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:20.817493 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:20.818010 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:20.818011 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:20.818353 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:20.818354 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:20.818393 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:20.818863 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:48:20.818868 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:48:20.820599 (+ 1731us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.159.190:43653: Server oryx/127.4.159.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":61,"threads_started":1}
W20260807 08:48:20.820767 18495 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:48:20.816450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.131:36719 (local address 127.4.159.190:43653)
0807 08:48:20.816496 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:20.816498 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:20.816538 (+ 40us) server_negotiation.cc:407] Connection header received
0807 08:48:20.816617 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:20.816617 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:20.816630 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:20.816670 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:20.816936 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:20.817434 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:20.818081 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:20.818278 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:20.818326 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:20.820699 (+ 2373us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.159.131:36719: BlockingRecv error: recv got EOF from 127.4.159.131:36719 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
Aug 07 08:48:21 dist-test-slave-6zbq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.159.129@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:48:21 dist-test-slave-6zbq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.159.129@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:48:21.569767 18499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:21.565506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:43653 (local address 127.4.159.129:53853)
0807 08:48:21.565627 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:21.565660 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:21.565711 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:21.565832 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:21.565833 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:21.565986 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:48:21.566036 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:21.566037 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:21.566707 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:21.566708 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:21.567234 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:21.567235 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:21.567525 (+ 290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:21.567526 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:21.567563 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:21.568016 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:48:21.568021 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:48:21.569677 (+ 1656us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.159.190:43653: Server oryx/127.4.159.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":64,"threads_started":1}
W20260807 08:48:21.569844 18502 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:48:21.565590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.129:53853 (local address 127.4.159.190:43653)
0807 08:48:21.565699 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:21.565701 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:21.565711 (+ 10us) server_negotiation.cc:407] Connection header received
0807 08:48:21.565755 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:21.565756 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:21.565766 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:21.565821 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:21.566117 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:21.566646 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:21.567295 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:21.567475 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:21.567515 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:21.569781 (+ 2266us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.159.129:53853: BlockingRecv error: recv got EOF from 127.4.159.129:53853 (error 108)
Metrics: {"server-negotiator.queue_time_us":88,"thread_start_us":47,"threads_started":1}
Aug 07 08:48:21 dist-test-slave-6zbq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.159.130@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:48:21 dist-test-slave-6zbq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.159.130@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:48:21.715685 18504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:21.711078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:43653 (local address 127.4.159.130:54371)
0807 08:48:21.711226 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:21.711238 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:21.711309 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:21.711427 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:21.711428 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:21.711610 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:48:21.711664 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:21.711667 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:21.712331 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:21.712333 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:21.712853 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:21.712855 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:21.713226 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:21.713227 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:21.713289 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:21.713751 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:48:21.713757 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:48:21.715604 (+ 1847us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.159.190:43653: Server oryx/127.4.159.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":72,"threads_started":1}
W20260807 08:48:21.715745 18502 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:48:21.711156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.130:54371 (local address 127.4.159.190:43653)
0807 08:48:21.711231 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:21.711233 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:21.711288 (+ 55us) server_negotiation.cc:407] Connection header received
0807 08:48:21.711361 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:21.711362 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:21.711374 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:21.711421 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:21.711746 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:21.712246 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:21.712942 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:21.713135 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:21.713209 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:21.715686 (+ 2477us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.159.130:54371: BlockingRecv error: recv got EOF from 127.4.159.130:54371 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 07 08:48:21 dist-test-slave-6zbq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.159.131@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:48:21 dist-test-slave-6zbq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.159.131@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:48:21.825407 18505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:21.821122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:43653 (local address 127.4.159.131:53861)
0807 08:48:21.821254 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:21.821265 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:21.821320 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:21.821455 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:21.821456 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:21.821623 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:48:21.821670 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:21.821672 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:21.822321 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:21.822323 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:21.822876 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:21.822879 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:21.823231 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:21.823232 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:21.823288 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:21.823700 (+ 412us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:48:21.823704 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:48:21.825335 (+ 1631us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.159.190:43653: Server oryx/127.4.159.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":55,"threads_started":1}
W20260807 08:48:21.825497 18502 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:48:21.821254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.131:53861 (local address 127.4.159.190:43653)
0807 08:48:21.821319 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:21.821322 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:21.821331 (+ 9us) server_negotiation.cc:407] Connection header received
0807 08:48:21.821375 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:21.821376 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:21.821391 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:21.821439 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:21.821748 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:21.822235 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:21.822962 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:21.823157 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:21.823213 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:21.825442 (+ 2229us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.159.131:53861: BlockingRecv error: recv got EOF from 127.4.159.131:53861 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 07 08:48:22 dist-test-slave-6zbq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.159.129@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:48:22 dist-test-slave-6zbq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.159.129@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:48:22.574617 18508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:22.570216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:43653 (local address 127.4.159.129:49077)
0807 08:48:22.570365 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:22.570377 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:22.570426 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:22.570552 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:22.570553 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:22.570708 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:48:22.570756 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:22.570757 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:22.571420 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:22.571421 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:22.571937 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:22.571939 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:22.572233 (+ 294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:22.572233 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:22.572268 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:22.572681 (+ 413us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:48:22.572685 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:48:22.574503 (+ 1818us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.159.190:43653: Server oryx/127.4.159.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":59,"threads_started":1}
W20260807 08:48:22.574705 18509 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:48:22.570276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.129:49077 (local address 127.4.159.190:43653)
0807 08:48:22.570399 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:22.570401 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:22.570413 (+ 12us) server_negotiation.cc:407] Connection header received
0807 08:48:22.570469 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:22.570469 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:22.570480 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:22.570539 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:22.570849 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:22.571359 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:22.571992 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:22.572180 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:22.572225 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:22.574643 (+ 2418us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.159.129:49077: BlockingRecv error: recv got EOF from 127.4.159.129:49077 (error 108)
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":83,"threads_started":1}
Aug 07 08:48:22 dist-test-slave-6zbq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.159.130@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:48:22 dist-test-slave-6zbq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.159.130@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:48:22.720589 18512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:22.716129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:43653 (local address 127.4.159.130:44925)
0807 08:48:22.716255 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:22.716265 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:22.716327 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:22.716448 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:22.716448 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:22.716611 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:48:22.716657 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:22.716659 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:22.717309 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:22.717310 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:22.717976 (+ 666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:22.717978 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:22.718326 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:22.718327 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:22.718415 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:22.718846 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:48:22.718851 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:48:22.720529 (+ 1678us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.159.190:43653: Server oryx/127.4.159.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":59,"threads_started":1}
W20260807 08:48:22.720685 18509 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:48:22.716182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.130:44925 (local address 127.4.159.190:43653)
0807 08:48:22.716259 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:22.716261 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:22.716322 (+ 61us) server_negotiation.cc:407] Connection header received
0807 08:48:22.716383 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:22.716384 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:22.716395 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:22.716431 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:22.716710 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:22.717249 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:22.718049 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:22.718253 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:22.718306 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:22.720622 (+ 2316us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.159.130:44925: BlockingRecv error: recv got EOF from 127.4.159.130:44925 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 07 08:48:22 dist-test-slave-6zbq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.159.131@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:48:22 dist-test-slave-6zbq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.159.131@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:48:22.830118 18516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:22.825850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:43653 (local address 127.4.159.131:43815)
0807 08:48:22.825984 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:22.825995 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:22.826056 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:22.826161 (+ 105us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:22.826162 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:22.826325 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:48:22.826388 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:22.826390 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:22.827003 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:22.827003 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:22.827522 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:22.827523 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:22.827822 (+ 299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:22.827827 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:22.827868 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:22.828355 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:48:22.828359 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:48:22.830046 (+ 1687us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.159.190:43653: Server oryx/127.4.159.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":75,"threads_started":1}
W20260807 08:48:22.830200 18509 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:48:22.825926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.131:43815 (local address 127.4.159.190:43653)
0807 08:48:22.826007 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:22.826009 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:22.826039 (+ 30us) server_negotiation.cc:407] Connection header received
0807 08:48:22.826097 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:22.826097 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:22.826111 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:22.826154 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:22.826452 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:22.826940 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:22.827584 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:22.827764 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:22.827811 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:22.830137 (+ 2326us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.159.131:43815: BlockingRecv error: recv got EOF from 127.4.159.131:43815 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 07 08:48:23 dist-test-slave-6zbq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.159.129@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:48:23 dist-test-slave-6zbq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.159.129@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:48:23.579245 18517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:23.575065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:43653 (local address 127.4.159.129:57167)
0807 08:48:23.575195 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:23.575207 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:23.575269 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:23.575401 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:23.575401 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:23.575557 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:48:23.575606 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:23.575608 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:23.576210 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:23.576210 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:23.576728 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:23.576729 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:23.577093 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:23.577094 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:23.577130 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:23.577545 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:48:23.577549 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:48:23.579185 (+ 1636us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.159.190:43653: Server oryx/127.4.159.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":58,"threads_started":1}
W20260807 08:48:23.579356 18521 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:48:23.575130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.129:57167 (local address 127.4.159.190:43653)
0807 08:48:23.575292 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:23.575294 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:23.575301 (+ 7us) server_negotiation.cc:407] Connection header received
0807 08:48:23.575322 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:23.575323 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:23.575334 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:23.575388 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:23.575669 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:23.576157 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:23.576825 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:23.577030 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:23.577093 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:23.579282 (+ 2189us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.159.129:57167: BlockingRecv error: recv got EOF from 127.4.159.129:57167 (error 108)
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":99,"threads_started":1}
Aug 07 08:48:23 dist-test-slave-6zbq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.159.130@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:48:23 dist-test-slave-6zbq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.159.130@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:48:23.725222 18522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:23.721019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:43653 (local address 127.4.159.130:40145)
0807 08:48:23.721190 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:23.721200 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:23.721258 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:23.721380 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:23.721381 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:23.721546 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:48:23.721592 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:23.721594 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:23.722192 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:23.722193 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:23.722754 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:23.722757 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:23.723078 (+ 321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:23.723079 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:23.723115 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:23.723507 (+ 392us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:48:23.723511 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:48:23.725131 (+ 1620us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.159.190:43653: Server oryx/127.4.159.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":105,"threads_started":1}
W20260807 08:48:23.725306 18521 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:48:23.721123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.130:40145 (local address 127.4.159.190:43653)
0807 08:48:23.721191 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:23.721193 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:23.721256 (+ 63us) server_negotiation.cc:407] Connection header received
0807 08:48:23.721313 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:23.721314 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:23.721326 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:23.721364 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:23.721646 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:23.722125 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:23.722828 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:23.723020 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:23.723070 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:23.725253 (+ 2183us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.159.130:40145: BlockingRecv error: recv got EOF from 127.4.159.130:40145 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 07 08:48:23 dist-test-slave-6zbq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.159.131@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:48:23 dist-test-slave-6zbq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.159.131@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:48:23.834877 18524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:23.830570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:43653 (local address 127.4.159.131:36155)
0807 08:48:23.830720 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:23.830732 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:23.830805 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:23.830943 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:23.830944 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:23.831112 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:48:23.831156 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:23.831158 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:23.831804 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:23.831805 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:23.832311 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:23.832312 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:23.832651 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:23.832652 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:23.832715 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:23.833155 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:48:23.833159 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:48:23.834800 (+ 1641us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.159.190:43653: Server oryx/127.4.159.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":63,"threads_started":1}
W20260807 08:48:23.834969 18521 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:48:23.830668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.131:36155 (local address 127.4.159.190:43653)
0807 08:48:23.830737 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:23.830739 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:23.830801 (+ 62us) server_negotiation.cc:407] Connection header received
0807 08:48:23.830856 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:23.830857 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:23.830871 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:23.830921 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:23.831241 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:23.831723 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:23.832399 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:23.832582 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:23.832636 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:23.834913 (+ 2277us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.159.131:36155: BlockingRecv error: recv got EOF from 127.4.159.131:36155 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 07 08:48:24 dist-test-slave-6zbq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.159.129@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:48:24 dist-test-slave-6zbq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.159.129@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:48:24.583973 18525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:24.579680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:43653 (local address 127.4.159.129:57633)
0807 08:48:24.579803 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:24.579813 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:24.579861 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:24.579998 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:24.579998 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:24.580152 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:48:24.580206 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:24.580207 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:24.580884 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:24.580884 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:24.581417 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:24.581418 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:24.581759 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:24.581759 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:24.581800 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:24.582222 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:48:24.582226 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:48:24.583915 (+ 1689us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.159.190:43653: Server oryx/127.4.159.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":62,"threads_started":1}
W20260807 08:48:24.584072 18526 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:48:24.579746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.129:57633 (local address 127.4.159.190:43653)
0807 08:48:24.579889 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:24.579891 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:24.579899 (+ 8us) server_negotiation.cc:407] Connection header received
0807 08:48:24.579919 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:24.579920 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:24.579930 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:24.579989 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:24.580303 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:24.580816 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:24.581489 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:24.581696 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:24.581748 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:24.583999 (+ 2251us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.159.129:57633: BlockingRecv error: recv got EOF from 127.4.159.129:57633 (error 108)
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":72,"threads_started":1}
Aug 07 08:48:24 dist-test-slave-6zbq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.159.130@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:48:24 dist-test-slave-6zbq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.159.130@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:48:24.730110 18527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:24.725690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:43653 (local address 127.4.159.130:56137)
0807 08:48:24.725843 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:24.725853 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:24.725901 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:24.726016 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:24.726017 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:24.726188 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:48:24.726234 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:24.726236 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:24.726849 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:24.726850 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:24.727377 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:24.727379 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:24.727744 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:24.727745 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:24.727783 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:24.728224 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:48:24.728228 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:48:24.730051 (+ 1823us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.159.190:43653: Server oryx/127.4.159.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":95,"threads_started":1}
W20260807 08:48:24.730206 18526 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:48:24.725830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.130:56137 (local address 127.4.159.190:43653)
0807 08:48:24.725882 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:24.725884 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:24.725892 (+ 8us) server_negotiation.cc:407] Connection header received
0807 08:48:24.725946 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:24.725946 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:24.725956 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:24.726001 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:24.726289 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:24.726790 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:24.727469 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:24.727676 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:24.727738 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:24.730143 (+ 2405us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.159.130:56137: BlockingRecv error: recv got EOF from 127.4.159.130:56137 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
Aug 07 08:48:24 dist-test-slave-6zbq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.159.131@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:48:24 dist-test-slave-6zbq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.159.131@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:48:24.839730 18528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:24.835353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:43653 (local address 127.4.159.131:48037)
0807 08:48:24.835491 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:24.835504 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:24.835574 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:24.835708 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:24.835709 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:24.835888 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:48:24.835932 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:24.835935 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:24.836592 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:24.836593 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:24.837129 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:24.837132 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:24.837478 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:24.837479 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:24.837536 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:24.837957 (+ 421us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:48:24.837961 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:48:24.839633 (+ 1672us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.159.190:43653: Server oryx/127.4.159.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":69,"threads_started":1}
W20260807 08:48:24.839792 18526 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:48:24.835457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.131:48037 (local address 127.4.159.190:43653)
0807 08:48:24.835521 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:24.835523 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:24.835554 (+ 31us) server_negotiation.cc:407] Connection header received
0807 08:48:24.835625 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:24.835626 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:24.835640 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:24.835691 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:24.836011 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:24.836511 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:24.837221 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:24.837406 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:24.837462 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:24.839739 (+ 2277us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.159.131:48037: BlockingRecv error: recv got EOF from 127.4.159.131:48037 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 07 08:48:25 dist-test-slave-6zbq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.159.129@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:48:25 dist-test-slave-6zbq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.159.129@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:48:25.588685 18532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:25.584428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:43653 (local address 127.4.159.129:53637)
0807 08:48:25.584562 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:25.584571 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:25.584627 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:25.584770 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:25.584770 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:25.584928 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:48:25.584978 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:25.584980 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:25.585592 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:25.585593 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:25.586135 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:25.586137 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:25.586464 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:25.586464 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:25.586504 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:25.586921 (+ 417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:48:25.586925 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:48:25.588617 (+ 1692us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.159.190:43653: Server oryx/127.4.159.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":66,"threads_started":1}
W20260807 08:48:25.588801 18534 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:48:25.584491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.129:53637 (local address 127.4.159.190:43653)
0807 08:48:25.584659 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:25.584660 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:25.584668 (+ 8us) server_negotiation.cc:407] Connection header received
0807 08:48:25.584689 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:25.584689 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:25.584700 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:25.584757 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:25.585039 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:25.585528 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:25.586204 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:25.586412 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:25.586463 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:25.588724 (+ 2261us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.159.129:53637: BlockingRecv error: recv got EOF from 127.4.159.129:53637 (error 108)
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":77,"threads_started":1}
Aug 07 08:48:25 dist-test-slave-6zbq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.159.130@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:48:25 dist-test-slave-6zbq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.159.130@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:48:25.738780 18535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:25.730590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:43653 (local address 127.4.159.130:41819)
0807 08:48:25.730715 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:25.730726 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:25.730774 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:25.730901 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:25.730901 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:25.731068 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:48:25.731118 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:25.731120 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:25.731821 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:25.731822 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:25.732441 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:25.732444 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:25.732905 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:25.732906 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:25.732953 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:25.737062 (+ 4109us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:48:25.737067 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:48:25.738720 (+ 1653us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.159.190:43653: Server oryx/127.4.159.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":71,"threads_started":1}
W20260807 08:48:25.738880 18534 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:48:25.730692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.130:41819 (local address 127.4.159.190:43653)
0807 08:48:25.730773 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:25.730774 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:25.730784 (+ 10us) server_negotiation.cc:407] Connection header received
0807 08:48:25.730814 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:25.730815 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:25.730827 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:25.730879 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:25.731219 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:25.731715 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:25.732543 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:25.732838 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:25.732890 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:25.738821 (+ 5931us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.159.130:41819: BlockingRecv error: recv got EOF from 127.4.159.130:41819 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 07 08:48:25 dist-test-slave-6zbq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.159.131@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:48:25 dist-test-slave-6zbq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.159.131@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:48:25.844296 18538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:25.840193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:43653 (local address 127.4.159.131:54179)
0807 08:48:25.840344 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:25.840355 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:25.840418 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:25.840541 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:25.840542 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:25.840716 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:48:25.840765 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:25.840766 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:25.841374 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:25.841374 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:25.841886 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:25.841888 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:25.842169 (+ 281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:25.842169 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:25.842205 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:25.842635 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:48:25.842639 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:48:25.844238 (+ 1599us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.159.190:43653: Server oryx/127.4.159.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":85,"threads_started":1}
W20260807 08:48:25.844393 18534 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:48:25.840258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.131:54179 (local address 127.4.159.190:43653)
0807 08:48:25.840317 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:25.840319 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:25.840413 (+ 94us) server_negotiation.cc:407] Connection header received
0807 08:48:25.840472 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:25.840473 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:25.840485 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:25.840527 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:25.840813 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:25.841315 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:25.841939 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:25.842118 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:25.842161 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:25.844332 (+ 2171us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.159.131:54179: BlockingRecv error: recv got EOF from 127.4.159.131:54179 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Aug 07 08:48:26 dist-test-slave-6zbq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.159.129@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:48:26 dist-test-slave-6zbq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.159.129@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:48:26.593518 18539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:26.589174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:43653 (local address 127.4.159.129:55959)
0807 08:48:26.589307 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:26.589317 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:26.589368 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:26.589486 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:26.589486 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:26.589641 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:48:26.589690 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:26.589692 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:26.590293 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:26.590293 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:26.590823 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:26.590825 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:26.591123 (+ 298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:26.591123 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:26.591159 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:26.591699 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:48:26.591704 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:48:26.593460 (+ 1756us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.159.190:43653: Server oryx/127.4.159.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":67,"threads_started":1}
W20260807 08:48:26.593621 18541 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:48:26.589253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.129:55959 (local address 127.4.159.190:43653)
0807 08:48:26.589355 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:26.589356 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:26.589364 (+ 8us) server_negotiation.cc:407] Connection header received
0807 08:48:26.589412 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:26.589412 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:26.589422 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:26.589476 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:26.589743 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:26.590239 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:26.590878 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:26.591071 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:26.591117 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:26.593546 (+ 2429us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.159.129:55959: BlockingRecv error: recv got EOF from 127.4.159.129:55959 (error 108)
Metrics: {"server-negotiator.queue_time_us":81,"thread_start_us":51,"threads_started":1}
Aug 07 08:48:26 dist-test-slave-6zbq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.159.130@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:48:26 dist-test-slave-6zbq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.159.130@KRBTEST.COM for oryx/127.4.159.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:48:26.743886 18542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:26.739304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:43653 (local address 127.4.159.130:44061)
0807 08:48:26.739455 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:26.739466 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:26.739526 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:26.739684 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:26.739685 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:26.739904 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:48:26.739966 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:26.739969 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:26.740648 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:26.740649 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:26.741225 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:26.741227 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:26.741557 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:26.741558 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:26.741594 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:26.742049 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:48:26.742054 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:48:26.743820 (+ 1766us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.159.190:43653: Server oryx/127.4.159.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":75,"threads_started":1}
W20260807 08:48:26.743973 18541 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:48:26.739393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.130:44061 (local address 127.4.159.190:43653)
0807 08:48:26.739459 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:26.739462 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:26.739525 (+ 63us) server_negotiation.cc:407] Connection header received
0807 08:48:26.739575 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:26.739576 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:26.739591 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:26.739658 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:26.740049 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:26.740556 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:26.741309 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:26.741502 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:26.741553 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:26.743913 (+ 2360us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.159.130:44061: BlockingRecv error: recv got EOF from 127.4.159.130:44061 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 08:48:26.771327 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 17777
I20260807 08:48:26.785111 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 18034
I20260807 08:48:26.795535 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 18246
I20260807 08:48:26.802755 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 17677
2026-08-07T08:48:26Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17477 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-07T08:48:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:48:26Z Disabled control of system clock
I20260807 08:48:26.832541 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786092453486433-4734-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786092453486433-4734-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786092453486433-4734-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.159.190:36527
--webserver_interface=127.4.159.190
--webserver_port=0
--builtin_ntp_servers=127.4.159.148:42983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.159.190:36527
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260807 08:48:26.912829 18547 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:48:26.913015 18547 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:48:26.913034 18547 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:48:26.914253 18547 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:48:26.914294 18547 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:48:26.914306 18547 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:48:26.914318 18547 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:48:26.914330 18547 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:48:26.915730 18547 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:42983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786092453486433-4734-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786092453486433-4734-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.159.190:36527
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.159.190:36527
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.159.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18547
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786092453486433-4734-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:48:26.915954 18547 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:48:26.916231 18547 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:48:26.921149 18557 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:48:26.921149 18552 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:48:26.921178 18555 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:48:26.921265 18547 server_base.cc:1034] running on GCE node
I20260807 08:48:26.921576 18547 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:48:26.921844 18547 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:48:26.922984 18547 hybrid_clock.cc:648] HybridClock initialized: now 1786092506922970 us; error 37 us; skew 500 ppm
I20260807 08:48:26.924083 18547 webserver.cc:460] Webserver started at http://127.4.159.190:39465/ using document root <none> and password file <none>
I20260807 08:48:26.924297 18547 fs_manager.cc:359] Metadata directory not provided
I20260807 08:48:26.924345 18547 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:48:26.924487 18547 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:48:26.925458 18547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786092453486433-4734-0/minicluster-data/master-0/data/instance:
uuid: "30619e1cbdb84dc0b364de6045a581e7"
format_stamp: "Formatted at 2026-08-07 08:48:26 on dist-test-slave-6zbq"
server_key: "943edbb83f09859d082856086ebc3eb4"
server_key_iv: "d219c0de8bc76e70f8bd727a85a47905"
server_key_version: "encryptionkey@0"
I20260807 08:48:26.925771 18547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786092453486433-4734-0/minicluster-data/master-0/wal/instance:
uuid: "30619e1cbdb84dc0b364de6045a581e7"
format_stamp: "Formatted at 2026-08-07 08:48:26 on dist-test-slave-6zbq"
server_key: "943edbb83f09859d082856086ebc3eb4"
server_key_iv: "d219c0de8bc76e70f8bd727a85a47905"
server_key_version: "encryptionkey@0"
I20260807 08:48:26.927038 18547 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 08:48:26.927721 18566 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:48:26.927908 18547 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 08:48:26.927990 18547 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786092453486433-4734-0/minicluster-data/master-0/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786092453486433-4734-0/minicluster-data/master-0/wal
uuid: "30619e1cbdb84dc0b364de6045a581e7"
format_stamp: "Formatted at 2026-08-07 08:48:26 on dist-test-slave-6zbq"
server_key: "943edbb83f09859d082856086ebc3eb4"
server_key_iv: "d219c0de8bc76e70f8bd727a85a47905"
server_key_version: "encryptionkey@0"
I20260807 08:48:26.928056 18547 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786092453486433-4734-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786092453486433-4734-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786092453486433-4734-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:48:26.942844 18547 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:48:26.943104 18547 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:48:26.943224 18547 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:48:26.946954 18547 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.190:36527
I20260807 08:48:26.946985 18659 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.190:36527 every 8 connection(s)
I20260807 08:48:26.947286 18547 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
I20260807 08:48:26.947923 18660 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:48:26.950430 18660 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30619e1cbdb84dc0b364de6045a581e7: Bootstrap starting.
I20260807 08:48:26.950994 18660 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 30619e1cbdb84dc0b364de6045a581e7: Neither blocks nor log segments found. Creating new log.
I20260807 08:48:26.951273 18660 log.cc:824] T 00000000000000000000000000000000 P 30619e1cbdb84dc0b364de6045a581e7: Log is configured to *not* fsync() on all Append() calls
I20260807 08:48:26.951910 18660 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30619e1cbdb84dc0b364de6045a581e7: No bootstrap required, opened a new log
I20260807 08:48:26.953444 18660 raft_consensus.cc:348] T 00000000000000000000000000000000 P 30619e1cbdb84dc0b364de6045a581e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30619e1cbdb84dc0b364de6045a581e7" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 36527 } }
I20260807 08:48:26.953576 18660 raft_consensus.cc:374] T 00000000000000000000000000000000 P 30619e1cbdb84dc0b364de6045a581e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:48:26.953608 18660 raft_consensus.cc:729] T 00000000000000000000000000000000 P 30619e1cbdb84dc0b364de6045a581e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30619e1cbdb84dc0b364de6045a581e7, State: Initialized, Role: FOLLOWER
I20260807 08:48:26.953752 18660 consensus_queue.cc:260] T 00000000000000000000000000000000 P 30619e1cbdb84dc0b364de6045a581e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30619e1cbdb84dc0b364de6045a581e7" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 36527 } }
I20260807 08:48:26.953814 18660 raft_consensus.cc:388] T 00000000000000000000000000000000 P 30619e1cbdb84dc0b364de6045a581e7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:48:26.953835 18660 raft_consensus.cc:482] T 00000000000000000000000000000000 P 30619e1cbdb84dc0b364de6045a581e7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:48:26.953881 18660 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 30619e1cbdb84dc0b364de6045a581e7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:48:26.954443 18660 raft_consensus.cc:504] T 00000000000000000000000000000000 P 30619e1cbdb84dc0b364de6045a581e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30619e1cbdb84dc0b364de6045a581e7" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 36527 } }
I20260807 08:48:26.954538 18660 leader_election.cc:302] T 00000000000000000000000000000000 P 30619e1cbdb84dc0b364de6045a581e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 30619e1cbdb84dc0b364de6045a581e7; no voters:
I20260807 08:48:26.954701 18660 leader_election.cc:288] T 00000000000000000000000000000000 P 30619e1cbdb84dc0b364de6045a581e7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:48:26.954735 18663 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 30619e1cbdb84dc0b364de6045a581e7 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:48:26.954917 18663 raft_consensus.cc:686] T 00000000000000000000000000000000 P 30619e1cbdb84dc0b364de6045a581e7 [term 1 LEADER]: Becoming Leader. State: Replica: 30619e1cbdb84dc0b364de6045a581e7, State: Running, Role: LEADER
I20260807 08:48:26.954926 18660 sys_catalog.cc:564] T 00000000000000000000000000000000 P 30619e1cbdb84dc0b364de6045a581e7 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:48:26.955056 18663 consensus_queue.cc:237] T 00000000000000000000000000000000 P 30619e1cbdb84dc0b364de6045a581e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30619e1cbdb84dc0b364de6045a581e7" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 36527 } }
I20260807 08:48:26.955439 18665 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30619e1cbdb84dc0b364de6045a581e7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 30619e1cbdb84dc0b364de6045a581e7. Latest consensus state: current_term: 1 leader_uuid: "30619e1cbdb84dc0b364de6045a581e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30619e1cbdb84dc0b364de6045a581e7" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 36527 } } }
I20260807 08:48:26.955531 18665 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30619e1cbdb84dc0b364de6045a581e7 [sys.catalog]: This master's current role is: LEADER
I20260807 08:48:26.955554 18664 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30619e1cbdb84dc0b364de6045a581e7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "30619e1cbdb84dc0b364de6045a581e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30619e1cbdb84dc0b364de6045a581e7" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 36527 } } }
I20260807 08:48:26.955629 18664 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30619e1cbdb84dc0b364de6045a581e7 [sys.catalog]: This master's current role is: LEADER
I20260807 08:48:26.956542 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 18547
I20260807 08:48:26.956635 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786092453486433-4734-0/minicluster-data/master-0/wal/instance
I20260807 08:48:26.956789 4734 external_mini_cluster.cc:1428] Setting key be14f1921523afb722027c224496149e
W20260807 08:48:26.957307 18689 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 30619e1cbdb84dc0b364de6045a581e7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 08:48:26.957372 18689 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 08:48:26.957455 18674 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:48:26.957645 18674 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:48:26.959182 18674 catalog_manager.cc:1347] Generated new cluster ID: d4ab6a3150744785807ac2e1a29ff25a
I20260807 08:48:26.959237 18674 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:48:26.963951 18696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:26.957242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48802 (local address 127.4.159.190:36527)
0807 08:48:26.958696 (+ 1454us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:26.958704 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:26.958730 (+ 26us) server_negotiation.cc:407] Connection header received
0807 08:48:26.958837 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:26.958844 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:26.958913 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:26.959049 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:26.959331 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:26.960690 (+ 1359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:26.961774 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:26.962412 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:26.962553 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:26.963226 (+ 673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:48:26.963230 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:48:26.963231 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:48:26.963456 (+ 225us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:48:26.963503 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:48:26.963521 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:48:26.963599 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 08:48:26.963644 (+ 45us) server_negotiation.cc:299] Negotiation successful
0807 08:48:26.963732 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1369,"thread_start_us":53,"threads_started":1}
I20260807 08:48:26.974642 18674 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:48:26.975097 18674 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:48:26.983906 18674 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 30619e1cbdb84dc0b364de6045a581e7: Generated new TSK 0
I20260807 08:48:26.984068 18674 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:48:26.987633 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786092453486433-4734-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786092453486433-4734-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786092453486433-4734-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786092453486433-4734-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.159.129:0
--local_ip_for_outbound_sockets=127.4.159.129
--webserver_interface=127.4.159.129
--webserver_port=0
--tserver_master_addrs=127.4.159.190:36527
--builtin_ntp_servers=127.4.159.148:42983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260807 08:48:27.093547 18698 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:48:27.093731 18698 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:48:27.093751 18698 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:48:27.094902 18698 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:48:27.094940 18698 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:48:27.094964 18698 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.159.129
I20260807 08:48:27.096254 18698 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:42983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786092453486433-4734-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786092453486433-4734-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.159.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786092453486433-4734-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.159.129
--webserver_port=0
--tserver_master_addrs=127.4.159.190:36527
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18698
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.159.129
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786092453486433-4734-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:48:27.096457 18698 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:48:27.096747 18698 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:48:27.102218 18704 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:48:27.102252 18705 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:48:27.102461 18708 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:48:27.102571 18698 server_base.cc:1034] running on GCE node
I20260807 08:48:27.102779 18698 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:48:27.103034 18698 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:48:27.104203 18698 hybrid_clock.cc:648] HybridClock initialized: now 1786092507104190 us; error 27 us; skew 500 ppm
I20260807 08:48:27.105661 18698 webserver.cc:460] Webserver started at http://127.4.159.129:33725/ using document root <none> and password file <none>
I20260807 08:48:27.105944 18698 fs_manager.cc:359] Metadata directory not provided
I20260807 08:48:27.106007 18698 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:48:27.106137 18698 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:48:27.107447 18698 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786092453486433-4734-0/minicluster-data/ts-0/data/instance:
uuid: "e6ad396ef5884e07ab7ad187c4c6d6f9"
format_stamp: "Formatted at 2026-08-07 08:48:27 on dist-test-slave-6zbq"
server_key: "43ba638b991722ffc94c56bb410484f7"
server_key_iv: "ea9fd02b70e58b2b20c9e97aa579fdb4"
server_key_version: "encryptionkey@0"
I20260807 08:48:27.107736 18698 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786092453486433-4734-0/minicluster-data/ts-0/wal/instance:
uuid: "e6ad396ef5884e07ab7ad187c4c6d6f9"
format_stamp: "Formatted at 2026-08-07 08:48:27 on dist-test-slave-6zbq"
server_key: "43ba638b991722ffc94c56bb410484f7"
server_key_iv: "ea9fd02b70e58b2b20c9e97aa579fdb4"
server_key_version: "encryptionkey@0"
I20260807 08:48:27.108923 18698 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 08:48:27.109598 18720 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:48:27.109746 18698 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 08:48:27.109818 18698 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786092453486433-4734-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786092453486433-4734-0/minicluster-data/ts-0/wal
uuid: "e6ad396ef5884e07ab7ad187c4c6d6f9"
format_stamp: "Formatted at 2026-08-07 08:48:27 on dist-test-slave-6zbq"
server_key: "43ba638b991722ffc94c56bb410484f7"
server_key_iv: "ea9fd02b70e58b2b20c9e97aa579fdb4"
server_key_version: "encryptionkey@0"
I20260807 08:48:27.109870 18698 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786092453486433-4734-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786092453486433-4734-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786092453486433-4734-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:48:27.122717 18698 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:48:27.122963 18698 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:48:27.123060 18698 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:48:27.123253 18698 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:48:27.123523 18698 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:48:27.123553 18698 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:48:27.123574 18698 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:48:27.123584 18698 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:48:27.129005 18698 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.129:37557
I20260807 08:48:27.129042 18892 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.129:37557 every 8 connection(s)
I20260807 08:48:27.129351 18698 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786092453486433-4734-0/minicluster-data/ts-0/data/info.pb
I20260807 08:48:27.133839 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 18698
I20260807 08:48:27.133941 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786092453486433-4734-0/minicluster-data/ts-0/wal/instance
I20260807 08:48:27.134122 4734 external_mini_cluster.cc:1428] Setting key 699049a1b33d08d5e3667c916b2eaedd
I20260807 08:48:27.134716 18696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:27.130003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.129:32895 (local address 127.4.159.190:36527)
0807 08:48:27.130109 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:27.130111 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:27.130280 (+ 169us) server_negotiation.cc:407] Connection header received
0807 08:48:27.130804 (+ 524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:27.130805 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:27.130818 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:27.130863 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:27.131533 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:27.132071 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:27.133091 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:27.133396 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:27.133466 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:27.134439 (+ 973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:48:27.134441 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:48:27.134441 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:48:27.134446 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:48:27.134463 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:48:27.134467 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:48:27.134519 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0807 08:48:27.134599 (+ 80us) server_negotiation.cc:299] Negotiation successful
0807 08:48:27.134628 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 08:48:27.134888 18895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:27.129981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:36527 (local address 127.4.159.129:32895)
0807 08:48:27.130168 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:27.130200 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:27.130719 (+ 519us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:27.130924 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:27.130927 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:27.130935 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:48:27.131463 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:27.131469 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:27.132170 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:27.132171 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:27.132973 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:27.132976 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:27.133511 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:27.133513 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:27.133653 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:27.134245 (+ 592us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 08:48:27.134246 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:48:27.134284 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 08:48:27.134312 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:48:27.134523 (+ 211us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:48:27.134529 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:48:27.134531 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 08:48:27.134591 (+ 60us) client_negotiation.cc:240] Negotiation successful
0807 08:48:27.134674 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":83,"threads_started":1}
I20260807 08:48:27.135154 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786092453486433-4734-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786092453486433-4734-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786092453486433-4734-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786092453486433-4734-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.159.130:0
--local_ip_for_outbound_sockets=127.4.159.130
--webserver_interface=127.4.159.130
--webserver_port=0
--tserver_master_addrs=127.4.159.190:36527
--builtin_ntp_servers=127.4.159.148:42983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260807 08:48:27.135336 18893 heartbeater.cc:347] Connected to a master server at 127.4.159.190:36527
I20260807 08:48:27.135421 18893 heartbeater.cc:464] Registering TS with master...
I20260807 08:48:27.135596 18893 heartbeater.cc:511] Master 127.4.159.190:36527 requested a full tablet report, sending...
I20260807 08:48:27.136065 18595 ts_manager.cc:194] Registered new tserver with Master: e6ad396ef5884e07ab7ad187c4c6d6f9 (127.4.159.129:37557)
I20260807 08:48:27.136677 18595 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.159.129:32895
W20260807 08:48:27.218561 18897 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:48:27.218747 18897 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:48:27.218766 18897 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:48:27.219836 18897 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:48:27.219871 18897 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:48:27.219894 18897 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.159.130
I20260807 08:48:27.221246 18897 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:42983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786092453486433-4734-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786092453486433-4734-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.159.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786092453486433-4734-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.159.130
--webserver_port=0
--tserver_master_addrs=127.4.159.190:36527
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18897
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.159.130
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786092453486433-4734-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:48:27.221443 18897 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:48:27.221688 18897 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:48:27.226393 18911 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:48:27.226406 18909 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:48:27.226406 18908 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:48:27.226435 18897 server_base.cc:1034] running on GCE node
I20260807 08:48:27.226729 18897 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:48:27.226935 18897 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:48:27.228070 18897 hybrid_clock.cc:648] HybridClock initialized: now 1786092507228062 us; error 27 us; skew 500 ppm
I20260807 08:48:27.229128 18897 webserver.cc:460] Webserver started at http://127.4.159.130:44177/ using document root <none> and password file <none>
I20260807 08:48:27.229314 18897 fs_manager.cc:359] Metadata directory not provided
I20260807 08:48:27.229353 18897 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:48:27.229444 18897 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:48:27.230463 18897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786092453486433-4734-0/minicluster-data/ts-1/data/instance:
uuid: "b157ef5643df43cba0e4b48af57a22dd"
format_stamp: "Formatted at 2026-08-07 08:48:27 on dist-test-slave-6zbq"
server_key: "4a8787a7bd9069ad0161b198dba77da1"
server_key_iv: "ba8406b997f578cd0c6caee58f4b61cd"
server_key_version: "encryptionkey@0"
I20260807 08:48:27.230821 18897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786092453486433-4734-0/minicluster-data/ts-1/wal/instance:
uuid: "b157ef5643df43cba0e4b48af57a22dd"
format_stamp: "Formatted at 2026-08-07 08:48:27 on dist-test-slave-6zbq"
server_key: "4a8787a7bd9069ad0161b198dba77da1"
server_key_iv: "ba8406b997f578cd0c6caee58f4b61cd"
server_key_version: "encryptionkey@0"
I20260807 08:48:27.232012 18897 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:48:27.232637 18918 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:48:27.232788 18897 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 08:48:27.232864 18897 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786092453486433-4734-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786092453486433-4734-0/minicluster-data/ts-1/wal
uuid: "b157ef5643df43cba0e4b48af57a22dd"
format_stamp: "Formatted at 2026-08-07 08:48:27 on dist-test-slave-6zbq"
server_key: "4a8787a7bd9069ad0161b198dba77da1"
server_key_iv: "ba8406b997f578cd0c6caee58f4b61cd"
server_key_version: "encryptionkey@0"
I20260807 08:48:27.232930 18897 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786092453486433-4734-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786092453486433-4734-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786092453486433-4734-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:48:27.250196 18897 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:48:27.250804 18897 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:48:27.251019 18897 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:48:27.251516 18897 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:48:27.252177 18897 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:48:27.252312 18897 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:48:27.252359 18897 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:48:27.252375 18897 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:48:27.258472 18897 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.130:35221
I20260807 08:48:27.258527 19103 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.130:35221 every 8 connection(s)
I20260807 08:48:27.258870 18897 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786092453486433-4734-0/minicluster-data/ts-1/data/info.pb
I20260807 08:48:27.260704 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 18897
I20260807 08:48:27.260782 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786092453486433-4734-0/minicluster-data/ts-1/wal/instance
I20260807 08:48:27.260957 4734 external_mini_cluster.cc:1428] Setting key 60adad8d97ba43872b4b9bb2f18d578b
I20260807 08:48:27.261875 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786092453486433-4734-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786092453486433-4734-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786092453486433-4734-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786092453486433-4734-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.159.131:0
--local_ip_for_outbound_sockets=127.4.159.131
--webserver_interface=127.4.159.131
--webserver_port=0
--tserver_master_addrs=127.4.159.190:36527
--builtin_ntp_servers=127.4.159.148:42983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260807 08:48:27.264246 18696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:27.259595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.130:55179 (local address 127.4.159.190:36527)
0807 08:48:27.259664 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:27.259667 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:27.259916 (+ 249us) server_negotiation.cc:407] Connection header received
0807 08:48:27.260397 (+ 481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:27.260398 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:27.260408 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:27.260459 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:27.261200 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:27.261797 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:27.262870 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:27.263067 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:27.263121 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:27.263998 (+ 877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:48:27.264000 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:48:27.264000 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:48:27.264004 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:48:27.264020 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:48:27.264024 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:48:27.264067 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0807 08:48:27.264162 (+ 95us) server_negotiation.cc:299] Negotiation successful
0807 08:48:27.264187 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260807 08:48:27.264477 19110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:27.259588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:36527 (local address 127.4.159.130:55179)
0807 08:48:27.259803 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:27.259838 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:27.260317 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:27.260534 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:27.260537 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:27.260546 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:48:27.261113 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:27.261119 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:27.261888 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:27.261889 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:27.262783 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:27.262785 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:27.263140 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:27.263141 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:27.263293 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:27.263843 (+ 550us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 08:48:27.263843 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:48:27.263882 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 08:48:27.263910 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:48:27.264081 (+ 171us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:48:27.264085 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:48:27.264088 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 08:48:27.264150 (+ 62us) client_negotiation.cc:240] Negotiation successful
0807 08:48:27.264256 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":105,"threads_started":1}
I20260807 08:48:27.264935 19107 heartbeater.cc:347] Connected to a master server at 127.4.159.190:36527
I20260807 08:48:27.265026 19107 heartbeater.cc:464] Registering TS with master...
I20260807 08:48:27.265204 19107 heartbeater.cc:511] Master 127.4.159.190:36527 requested a full tablet report, sending...
I20260807 08:48:27.265570 18595 ts_manager.cc:194] Registered new tserver with Master: b157ef5643df43cba0e4b48af57a22dd (127.4.159.130:35221)
I20260807 08:48:27.265936 18595 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.159.130:55179
W20260807 08:48:27.347609 19112 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:48:27.347805 19112 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:48:27.347823 19112 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:48:27.348929 19112 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:48:27.348969 19112 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:48:27.348994 19112 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.159.131
I20260807 08:48:27.350291 19112 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:42983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786092453486433-4734-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786092453486433-4734-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.159.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786092453486433-4734-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.159.131
--webserver_port=0
--tserver_master_addrs=127.4.159.190:36527
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19112
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.159.131
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786092453486433-4734-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:48:27.350531 19112 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:48:27.350816 19112 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:48:27.355407 19120 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:48:27.355470 19122 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:48:27.355499 19119 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:48:27.355695 19112 server_base.cc:1034] running on GCE node
I20260807 08:48:27.355878 19112 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:48:27.356133 19112 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:48:27.357271 19112 hybrid_clock.cc:648] HybridClock initialized: now 1786092507357258 us; error 28 us; skew 500 ppm
I20260807 08:48:27.358536 19112 webserver.cc:460] Webserver started at http://127.4.159.131:37841/ using document root <none> and password file <none>
I20260807 08:48:27.358742 19112 fs_manager.cc:359] Metadata directory not provided
I20260807 08:48:27.358791 19112 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:48:27.358923 19112 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:48:27.359906 19112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786092453486433-4734-0/minicluster-data/ts-2/data/instance:
uuid: "de6fd9ed52d945ec9add05e549aaf03e"
format_stamp: "Formatted at 2026-08-07 08:48:27 on dist-test-slave-6zbq"
server_key: "c1dc3bb751fbb75e6eb23db917996a38"
server_key_iv: "b2ca754199fac8eb204c86a6cd9b12c4"
server_key_version: "encryptionkey@0"
I20260807 08:48:27.360237 19112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786092453486433-4734-0/minicluster-data/ts-2/wal/instance:
uuid: "de6fd9ed52d945ec9add05e549aaf03e"
format_stamp: "Formatted at 2026-08-07 08:48:27 on dist-test-slave-6zbq"
server_key: "c1dc3bb751fbb75e6eb23db917996a38"
server_key_iv: "b2ca754199fac8eb204c86a6cd9b12c4"
server_key_version: "encryptionkey@0"
I20260807 08:48:27.361457 19112 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 08:48:27.362146 19129 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:48:27.362331 19112 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 08:48:27.362442 19112 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786092453486433-4734-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786092453486433-4734-0/minicluster-data/ts-2/wal
uuid: "de6fd9ed52d945ec9add05e549aaf03e"
format_stamp: "Formatted at 2026-08-07 08:48:27 on dist-test-slave-6zbq"
server_key: "c1dc3bb751fbb75e6eb23db917996a38"
server_key_iv: "b2ca754199fac8eb204c86a6cd9b12c4"
server_key_version: "encryptionkey@0"
I20260807 08:48:27.362505 19112 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786092453486433-4734-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786092453486433-4734-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786092453486433-4734-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:48:27.372017 19112 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:48:27.372256 19112 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:48:27.372382 19112 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:48:27.372565 19112 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:48:27.372851 19112 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:48:27.372884 19112 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:48:27.372910 19112 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:48:27.372926 19112 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:48:27.378268 19112 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.131:36115
I20260807 08:48:27.378315 19297 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.131:36115 every 8 connection(s)
I20260807 08:48:27.378630 19112 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786092453486433-4734-0/minicluster-data/ts-2/data/info.pb
I20260807 08:48:27.383140 18696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:27.379255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.131:45189 (local address 127.4.159.190:36527)
0807 08:48:27.379306 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:27.379308 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:27.379516 (+ 208us) server_negotiation.cc:407] Connection header received
0807 08:48:27.380005 (+ 489us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:27.380006 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:27.380015 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:27.380061 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:27.380640 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:27.381131 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:27.382004 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:27.382183 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:27.382225 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:27.382938 (+ 713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:48:27.382939 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:48:27.382940 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:48:27.382945 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:48:27.382960 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:48:27.382964 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:48:27.383002 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0807 08:48:27.383056 (+ 54us) server_negotiation.cc:299] Negotiation successful
0807 08:48:27.383082 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260807 08:48:27.383309 19300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:27.379227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:36527 (local address 127.4.159.131:45189)
0807 08:48:27.379404 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:27.379441 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:27.379910 (+ 469us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:27.380122 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:27.380125 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:27.380133 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:48:27.380578 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:27.380582 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:27.381196 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:27.381196 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:27.381945 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:27.381947 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:27.382236 (+ 289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:27.382236 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:27.382335 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:27.382804 (+ 469us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 08:48:27.382804 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:48:27.382840 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 08:48:27.382863 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:48:27.383009 (+ 146us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:48:27.383012 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:48:27.383014 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 08:48:27.383057 (+ 43us) client_negotiation.cc:240] Negotiation successful
0807 08:48:27.383130 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":66,"threads_started":1}
I20260807 08:48:27.383749 19298 heartbeater.cc:347] Connected to a master server at 127.4.159.190:36527
I20260807 08:48:27.383829 19298 heartbeater.cc:464] Registering TS with master...
I20260807 08:48:27.383996 19298 heartbeater.cc:511] Master 127.4.159.190:36527 requested a full tablet report, sending...
I20260807 08:48:27.384338 18595 ts_manager.cc:194] Registered new tserver with Master: de6fd9ed52d945ec9add05e549aaf03e (127.4.159.131:36115)
I20260807 08:48:27.384683 18595 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.159.131:45189
I20260807 08:48:27.388002 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 19112
I20260807 08:48:27.388085 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786092453486433-4734-0/minicluster-data/ts-2/wal/instance
I20260807 08:48:27.388257 4734 external_mini_cluster.cc:1428] Setting key ebf6119d7bd19d74449817933db34012
I20260807 08:48:27.388998 4734 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260807 08:48:27.390694 18696 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:48:27.390206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48816 (local address 127.4.159.190:36527)
0807 08:48:27.390261 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:27.390263 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:27.390383 (+ 120us) server_negotiation.cc:407] Connection header received
0807 08:48:27.390456 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:27.390457 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:27.390466 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:27.390507 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:27.390620 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48816: BlockingRecv error: recv got EOF from 127.0.0.1:48816 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
I20260807 08:48:27.390731 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 18698
I20260807 08:48:27.408675 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 18897
I20260807 08:48:27.416152 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 19112
I20260807 08:48:27.423488 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 18547
2026-08-07T08:48:27Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (618 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-07T08:48:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:48:27Z Disabled control of system clock
I20260807 08:48:27.450913 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786092453486433-4734-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786092453486433-4734-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786092453486433-4734-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.159.190:45807
--webserver_interface=127.4.159.190
--webserver_port=0
--builtin_ntp_servers=127.4.159.148:43733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.159.190:45807
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260807 08:48:27.530526 19312 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:48:27.530712 19312 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:48:27.530732 19312 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:48:27.531770 19312 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:48:27.531801 19312 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:48:27.531812 19312 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:48:27.531823 19312 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:48:27.531833 19312 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:48:27.533072 19312 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:43733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786092453486433-4734-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786092453486433-4734-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.159.190:45807
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.159.190:45807
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.159.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19312
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786092453486433-4734-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:48:27.533262 19312 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:48:27.533476 19312 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:48:27.537897 19322 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:48:27.537851 19319 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:48:27.538043 19312 server_base.cc:1034] running on GCE node
W20260807 08:48:27.538165 19318 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:48:27.538424 19312 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:48:27.538702 19312 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:48:27.539844 19312 hybrid_clock.cc:648] HybridClock initialized: now 1786092507539829 us; error 26 us; skew 500 ppm
I20260807 08:48:27.540949 19312 webserver.cc:460] Webserver started at http://127.4.159.190:46867/ using document root <none> and password file <none>
I20260807 08:48:27.541172 19312 fs_manager.cc:359] Metadata directory not provided
I20260807 08:48:27.541222 19312 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:48:27.541350 19312 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:48:27.542383 19312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786092453486433-4734-0/minicluster-data/master-0/data/instance:
uuid: "7942bdabdec849fc8671d52763a96a45"
format_stamp: "Formatted at 2026-08-07 08:48:27 on dist-test-slave-6zbq"
server_key: "733a712ff17308ff67ccb9b6592de8b6"
server_key_iv: "c00b68488d6fe1269e51cc5d490f5915"
server_key_version: "encryptionkey@0"
I20260807 08:48:27.542737 19312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786092453486433-4734-0/minicluster-data/master-0/wal/instance:
uuid: "7942bdabdec849fc8671d52763a96a45"
format_stamp: "Formatted at 2026-08-07 08:48:27 on dist-test-slave-6zbq"
server_key: "733a712ff17308ff67ccb9b6592de8b6"
server_key_iv: "c00b68488d6fe1269e51cc5d490f5915"
server_key_version: "encryptionkey@0"
I20260807 08:48:27.543926 19312 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 08:48:27.544580 19334 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:48:27.544736 19312 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 08:48:27.544842 19312 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786092453486433-4734-0/minicluster-data/master-0/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786092453486433-4734-0/minicluster-data/master-0/wal
uuid: "7942bdabdec849fc8671d52763a96a45"
format_stamp: "Formatted at 2026-08-07 08:48:27 on dist-test-slave-6zbq"
server_key: "733a712ff17308ff67ccb9b6592de8b6"
server_key_iv: "c00b68488d6fe1269e51cc5d490f5915"
server_key_version: "encryptionkey@0"
I20260807 08:48:27.544914 19312 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786092453486433-4734-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786092453486433-4734-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786092453486433-4734-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:48:27.548122 19312 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:48:27.548362 19312 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:48:27.548476 19312 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:48:27.552140 19312 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.190:45807
I20260807 08:48:27.552193 19421 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.190:45807 every 8 connection(s)
I20260807 08:48:27.552455 19312 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
I20260807 08:48:27.553052 19422 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:48:27.555006 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 19312
I20260807 08:48:27.555087 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786092453486433-4734-0/minicluster-data/master-0/wal/instance
I20260807 08:48:27.555347 4734 external_mini_cluster.cc:1428] Setting key 59105b05db5922d54de6939c7307c29c
I20260807 08:48:27.555945 19422 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7942bdabdec849fc8671d52763a96a45: Bootstrap starting.
I20260807 08:48:27.556640 19422 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7942bdabdec849fc8671d52763a96a45: Neither blocks nor log segments found. Creating new log.
I20260807 08:48:27.556965 19422 log.cc:824] T 00000000000000000000000000000000 P 7942bdabdec849fc8671d52763a96a45: Log is configured to *not* fsync() on all Append() calls
I20260807 08:48:27.557906 19422 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7942bdabdec849fc8671d52763a96a45: No bootstrap required, opened a new log
I20260807 08:48:27.559372 19428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:27.555812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36058 (local address 127.4.159.190:45807)
0807 08:48:27.556014 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:27.556022 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:27.556066 (+ 44us) server_negotiation.cc:407] Connection header received
0807 08:48:27.556582 (+ 516us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:27.556591 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:27.556644 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:27.556804 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:27.558660 (+ 1856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:48:27.558664 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:48:27.558664 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:48:27.558670 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:48:27.558749 (+ 79us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:48:27.558766 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:48:27.558894 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0807 08:48:27.558971 (+ 77us) server_negotiation.cc:299] Negotiation successful
0807 08:48:27.559052 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":79,"threads_started":1}
I20260807 08:48:27.559785 19422 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7942bdabdec849fc8671d52763a96a45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7942bdabdec849fc8671d52763a96a45" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 45807 } }
I20260807 08:48:27.559954 19422 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7942bdabdec849fc8671d52763a96a45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:48:27.559988 19422 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7942bdabdec849fc8671d52763a96a45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7942bdabdec849fc8671d52763a96a45, State: Initialized, Role: FOLLOWER
I20260807 08:48:27.560086 19422 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7942bdabdec849fc8671d52763a96a45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7942bdabdec849fc8671d52763a96a45" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 45807 } }
I20260807 08:48:27.560518 19422 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7942bdabdec849fc8671d52763a96a45 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:48:27.560557 19422 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7942bdabdec849fc8671d52763a96a45 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:48:27.560590 19422 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7942bdabdec849fc8671d52763a96a45 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:48:27.561053 19422 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7942bdabdec849fc8671d52763a96a45 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7942bdabdec849fc8671d52763a96a45" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 45807 } }
I20260807 08:48:27.561159 19422 leader_election.cc:302] T 00000000000000000000000000000000 P 7942bdabdec849fc8671d52763a96a45 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7942bdabdec849fc8671d52763a96a45; no voters:
I20260807 08:48:27.561280 19422 leader_election.cc:288] T 00000000000000000000000000000000 P 7942bdabdec849fc8671d52763a96a45 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:48:27.561331 19430 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7942bdabdec849fc8671d52763a96a45 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:48:27.561470 19430 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7942bdabdec849fc8671d52763a96a45 [term 1 LEADER]: Becoming Leader. State: Replica: 7942bdabdec849fc8671d52763a96a45, State: Running, Role: LEADER
I20260807 08:48:27.561506 19422 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7942bdabdec849fc8671d52763a96a45 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:48:27.561571 19430 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7942bdabdec849fc8671d52763a96a45 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7942bdabdec849fc8671d52763a96a45" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 45807 } }
I20260807 08:48:27.561914 19432 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7942bdabdec849fc8671d52763a96a45 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7942bdabdec849fc8671d52763a96a45" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7942bdabdec849fc8671d52763a96a45" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 45807 } } }
I20260807 08:48:27.561918 19433 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7942bdabdec849fc8671d52763a96a45 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7942bdabdec849fc8671d52763a96a45. Latest consensus state: current_term: 1 leader_uuid: "7942bdabdec849fc8671d52763a96a45" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7942bdabdec849fc8671d52763a96a45" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 45807 } } }
I20260807 08:48:27.562059 19432 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7942bdabdec849fc8671d52763a96a45 [sys.catalog]: This master's current role is: LEADER
I20260807 08:48:27.562070 19433 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7942bdabdec849fc8671d52763a96a45 [sys.catalog]: This master's current role is: LEADER
W20260807 08:48:27.563366 19458 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7942bdabdec849fc8671d52763a96a45: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 08:48:27.563411 19458 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 08:48:27.563470 19442 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:48:27.563620 19442 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:48:27.565065 19442 catalog_manager.cc:1347] Generated new cluster ID: 554fc586eb934220be0912ab1247d2d2
I20260807 08:48:27.565109 19442 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:48:27.573701 19442 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:48:27.573873 19442 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:48:27.581887 19442 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7942bdabdec849fc8671d52763a96a45: Generated new TSK 0
I20260807 08:48:27.582048 19442 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:48:27.588392 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786092453486433-4734-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786092453486433-4734-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786092453486433-4734-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786092453486433-4734-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.159.129:0
--local_ip_for_outbound_sockets=127.4.159.129
--webserver_interface=127.4.159.129
--webserver_port=0
--tserver_master_addrs=127.4.159.190:45807
--builtin_ntp_servers=127.4.159.148:43733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260807 08:48:27.672472 19461 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:48:27.672663 19461 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:48:27.672684 19461 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:48:27.673856 19461 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:48:27.673895 19461 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:48:27.673919 19461 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.159.129
I20260807 08:48:27.675295 19461 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:43733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786092453486433-4734-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786092453486433-4734-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.159.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786092453486433-4734-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.159.129
--webserver_port=0
--tserver_master_addrs=127.4.159.190:45807
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19461
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.159.129
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786092453486433-4734-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:48:27.675513 19461 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:48:27.675798 19461 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:48:27.680482 19471 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:48:27.680469 19474 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:48:27.680478 19470 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:48:27.680891 19461 server_base.cc:1034] running on GCE node
I20260807 08:48:27.681102 19461 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:48:27.681336 19461 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:48:27.682479 19461 hybrid_clock.cc:648] HybridClock initialized: now 1786092507682463 us; error 31 us; skew 500 ppm
I20260807 08:48:27.687282 19461 webserver.cc:460] Webserver started at http://127.4.159.129:32927/ using document root <none> and password file <none>
I20260807 08:48:27.687520 19461 fs_manager.cc:359] Metadata directory not provided
I20260807 08:48:27.687563 19461 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:48:27.687680 19461 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:48:27.688727 19461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786092453486433-4734-0/minicluster-data/ts-0/data/instance:
uuid: "069dd21b12fb4a5484e6569e6135edb4"
format_stamp: "Formatted at 2026-08-07 08:48:27 on dist-test-slave-6zbq"
server_key: "e6f5ad107354421c24e2c13916eea719"
server_key_iv: "918d5615fce93fd4972a8f0875833444"
server_key_version: "encryptionkey@0"
I20260807 08:48:27.689054 19461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786092453486433-4734-0/minicluster-data/ts-0/wal/instance:
uuid: "069dd21b12fb4a5484e6569e6135edb4"
format_stamp: "Formatted at 2026-08-07 08:48:27 on dist-test-slave-6zbq"
server_key: "e6f5ad107354421c24e2c13916eea719"
server_key_iv: "918d5615fce93fd4972a8f0875833444"
server_key_version: "encryptionkey@0"
I20260807 08:48:27.690289 19461 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260807 08:48:27.690990 19483 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:48:27.691139 19461 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 08:48:27.691233 19461 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786092453486433-4734-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786092453486433-4734-0/minicluster-data/ts-0/wal
uuid: "069dd21b12fb4a5484e6569e6135edb4"
format_stamp: "Formatted at 2026-08-07 08:48:27 on dist-test-slave-6zbq"
server_key: "e6f5ad107354421c24e2c13916eea719"
server_key_iv: "918d5615fce93fd4972a8f0875833444"
server_key_version: "encryptionkey@0"
I20260807 08:48:27.691289 19461 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786092453486433-4734-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786092453486433-4734-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786092453486433-4734-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:48:27.694569 19461 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:48:27.694789 19461 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:48:27.694880 19461 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:48:27.695034 19461 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:48:27.695286 19461 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:48:27.695322 19461 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:48:27.695339 19461 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:48:27.695350 19461 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:48:27.700667 19461 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.129:45733
I20260807 08:48:27.700738 19657 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.129:45733 every 8 connection(s)
I20260807 08:48:27.700989 19461 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786092453486433-4734-0/minicluster-data/ts-0/data/info.pb
I20260807 08:48:27.703727 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 19461
I20260807 08:48:27.703825 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786092453486433-4734-0/minicluster-data/ts-0/wal/instance
I20260807 08:48:27.703841 19428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:27.701673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.129:46943 (local address 127.4.159.190:45807)
0807 08:48:27.701734 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:27.701737 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:27.701947 (+ 210us) server_negotiation.cc:407] Connection header received
0807 08:48:27.702420 (+ 473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:27.702421 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:27.702428 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:27.702464 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:27.703571 (+ 1107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:48:27.703573 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:48:27.703573 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:48:27.703577 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:48:27.703594 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:48:27.703598 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:48:27.703651 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0807 08:48:27.703752 (+ 101us) server_negotiation.cc:299] Negotiation successful
0807 08:48:27.703764 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260807 08:48:27.704030 4734 external_mini_cluster.cc:1428] Setting key ccdf873a597e68360ec8eb133cc48d33
I20260807 08:48:27.704027 19660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:27.701663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:45807 (local address 127.4.159.129:46943)
0807 08:48:27.701840 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:27.701872 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:27.702299 (+ 427us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:27.702528 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:27.702531 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:27.702539 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:48:27.703391 (+ 852us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 08:48:27.703392 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:48:27.703419 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 08:48:27.703448 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:48:27.703668 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:48:27.703674 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:48:27.703677 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 08:48:27.703755 (+ 78us) client_negotiation.cc:240] Negotiation successful
0807 08:48:27.703822 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":87,"threads_started":1}
I20260807 08:48:27.704622 19658 heartbeater.cc:347] Connected to a master server at 127.4.159.190:45807
I20260807 08:48:27.704840 19658 heartbeater.cc:464] Registering TS with master...
I20260807 08:48:27.705062 19658 heartbeater.cc:511] Master 127.4.159.190:45807 requested a full tablet report, sending...
I20260807 08:48:27.705150 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786092453486433-4734-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786092453486433-4734-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786092453486433-4734-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786092453486433-4734-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.159.130:0
--local_ip_for_outbound_sockets=127.4.159.130
--webserver_interface=127.4.159.130
--webserver_port=0
--tserver_master_addrs=127.4.159.190:45807
--builtin_ntp_servers=127.4.159.148:43733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260807 08:48:27.708485 19363 ts_manager.cc:194] Registered new tserver with Master: 069dd21b12fb4a5484e6569e6135edb4 (127.4.159.129:45733)
W20260807 08:48:27.795441 19662 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:48:27.795697 19662 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:48:27.795733 19662 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:48:27.797359 19662 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:48:27.797410 19662 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:48:27.797447 19662 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.159.130
I20260807 08:48:27.799830 19662 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:43733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786092453486433-4734-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786092453486433-4734-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.159.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786092453486433-4734-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.159.130
--webserver_port=0
--tserver_master_addrs=127.4.159.190:45807
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19662
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.159.130
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786092453486433-4734-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:48:27.800900 19662 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:48:27.801216 19662 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:48:27.807020 19669 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:48:27.807013 19667 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:48:27.807030 19674 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:48:27.811352 19662 server_base.cc:1034] running on GCE node
I20260807 08:48:27.811574 19662 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:48:27.811836 19662 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:48:27.812971 19662 hybrid_clock.cc:648] HybridClock initialized: now 1786092507812947 us; error 44 us; skew 500 ppm
I20260807 08:48:27.814356 19662 webserver.cc:460] Webserver started at http://127.4.159.130:35077/ using document root <none> and password file <none>
I20260807 08:48:27.814641 19662 fs_manager.cc:359] Metadata directory not provided
I20260807 08:48:27.814702 19662 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:48:27.814838 19662 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:48:27.816015 19662 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786092453486433-4734-0/minicluster-data/ts-1/data/instance:
uuid: "a99467a768734103bb4b35daac645b58"
format_stamp: "Formatted at 2026-08-07 08:48:27 on dist-test-slave-6zbq"
server_key: "3ed927ce152d0aa85d3edae04e8011ce"
server_key_iv: "052d3a84eebd048b9669f3d28523f1d5"
server_key_version: "encryptionkey@0"
I20260807 08:48:27.816380 19662 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786092453486433-4734-0/minicluster-data/ts-1/wal/instance:
uuid: "a99467a768734103bb4b35daac645b58"
format_stamp: "Formatted at 2026-08-07 08:48:27 on dist-test-slave-6zbq"
server_key: "3ed927ce152d0aa85d3edae04e8011ce"
server_key_iv: "052d3a84eebd048b9669f3d28523f1d5"
server_key_version: "encryptionkey@0"
I20260807 08:48:27.817682 19662 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 08:48:27.818435 19685 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:48:27.818590 19662 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:48:27.818679 19662 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786092453486433-4734-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786092453486433-4734-0/minicluster-data/ts-1/wal
uuid: "a99467a768734103bb4b35daac645b58"
format_stamp: "Formatted at 2026-08-07 08:48:27 on dist-test-slave-6zbq"
server_key: "3ed927ce152d0aa85d3edae04e8011ce"
server_key_iv: "052d3a84eebd048b9669f3d28523f1d5"
server_key_version: "encryptionkey@0"
I20260807 08:48:27.818747 19662 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786092453486433-4734-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786092453486433-4734-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786092453486433-4734-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:48:27.821996 19662 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:48:27.822227 19662 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:48:27.822387 19662 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:48:27.822587 19662 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:48:27.822870 19662 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:48:27.822903 19662 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:48:27.822929 19662 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:48:27.822947 19662 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:48:27.828151 19662 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.130:32829
I20260807 08:48:27.828218 19877 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.130:32829 every 8 connection(s)
I20260807 08:48:27.828480 19662 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786092453486433-4734-0/minicluster-data/ts-1/data/info.pb
I20260807 08:48:27.828613 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 19662
I20260807 08:48:27.828714 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786092453486433-4734-0/minicluster-data/ts-1/wal/instance
I20260807 08:48:27.828917 4734 external_mini_cluster.cc:1428] Setting key 14f30de43f0720827714f0ca64aa3be4
I20260807 08:48:27.830054 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786092453486433-4734-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786092453486433-4734-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786092453486433-4734-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786092453486433-4734-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.159.131:0
--local_ip_for_outbound_sockets=127.4.159.131
--webserver_interface=127.4.159.131
--webserver_port=0
--tserver_master_addrs=127.4.159.190:45807
--builtin_ntp_servers=127.4.159.148:43733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260807 08:48:27.831326 19428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:27.829193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.130:40855 (local address 127.4.159.190:45807)
0807 08:48:27.829251 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:27.829253 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:27.829476 (+ 223us) server_negotiation.cc:407] Connection header received
0807 08:48:27.829937 (+ 461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:27.829939 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:27.829945 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:27.830012 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:27.831093 (+ 1081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:48:27.831094 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:48:27.831095 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:48:27.831100 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:48:27.831118 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:48:27.831122 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:48:27.831186 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 08:48:27.831238 (+ 52us) server_negotiation.cc:299] Negotiation successful
0807 08:48:27.831251 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260807 08:48:27.831451 19882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:27.829200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:45807 (local address 127.4.159.130:40855)
0807 08:48:27.829364 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:27.829394 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:27.829843 (+ 449us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:27.830061 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:27.830064 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:27.830071 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:48:27.830949 (+ 878us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 08:48:27.830950 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:48:27.830976 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 08:48:27.831003 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:48:27.831191 (+ 188us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:48:27.831193 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:48:27.831195 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 08:48:27.831242 (+ 47us) client_negotiation.cc:240] Negotiation successful
0807 08:48:27.831286 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":77,"threads_started":1}
I20260807 08:48:27.831925 19878 heartbeater.cc:347] Connected to a master server at 127.4.159.190:45807
I20260807 08:48:27.832008 19878 heartbeater.cc:464] Registering TS with master...
I20260807 08:48:27.832141 19878 heartbeater.cc:511] Master 127.4.159.190:45807 requested a full tablet report, sending...
I20260807 08:48:27.832499 19363 ts_manager.cc:194] Registered new tserver with Master: a99467a768734103bb4b35daac645b58 (127.4.159.130:32829)
W20260807 08:48:27.909716 19884 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:48:27.909905 19884 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:48:27.909925 19884 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:48:27.911108 19884 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:48:27.911146 19884 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:48:27.911170 19884 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.159.131
I20260807 08:48:27.912494 19884 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:43733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786092453486433-4734-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786092453486433-4734-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.159.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786092453486433-4734-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.159.131
--webserver_port=0
--tserver_master_addrs=127.4.159.190:45807
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19884
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.159.131
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786092453486433-4734-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:48:27.912716 19884 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:48:27.912992 19884 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:48:27.917615 19891 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:48:27.917650 19893 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:48:27.917753 19884 server_base.cc:1034] running on GCE node
W20260807 08:48:27.917853 19895 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:48:27.918053 19884 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:48:27.918280 19884 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:48:27.919417 19884 hybrid_clock.cc:648] HybridClock initialized: now 1786092507919415 us; error 39 us; skew 500 ppm
I20260807 08:48:27.920604 19884 webserver.cc:460] Webserver started at http://127.4.159.131:40875/ using document root <none> and password file <none>
I20260807 08:48:27.920830 19884 fs_manager.cc:359] Metadata directory not provided
I20260807 08:48:27.920878 19884 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:48:27.921005 19884 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:48:27.922080 19884 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786092453486433-4734-0/minicluster-data/ts-2/data/instance:
uuid: "ad5cc22c6b8447288612d75325910ca2"
format_stamp: "Formatted at 2026-08-07 08:48:27 on dist-test-slave-6zbq"
server_key: "2cd5e47bb3bc55b02905d11f3464c6ee"
server_key_iv: "78a1f993f7bf88510b0dfe0354cdd2b9"
server_key_version: "encryptionkey@0"
I20260807 08:48:27.922473 19884 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786092453486433-4734-0/minicluster-data/ts-2/wal/instance:
uuid: "ad5cc22c6b8447288612d75325910ca2"
format_stamp: "Formatted at 2026-08-07 08:48:27 on dist-test-slave-6zbq"
server_key: "2cd5e47bb3bc55b02905d11f3464c6ee"
server_key_iv: "78a1f993f7bf88510b0dfe0354cdd2b9"
server_key_version: "encryptionkey@0"
I20260807 08:48:27.923744 19884 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 08:48:27.924383 19904 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:48:27.924539 19884 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 08:48:27.924619 19884 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786092453486433-4734-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786092453486433-4734-0/minicluster-data/ts-2/wal
uuid: "ad5cc22c6b8447288612d75325910ca2"
format_stamp: "Formatted at 2026-08-07 08:48:27 on dist-test-slave-6zbq"
server_key: "2cd5e47bb3bc55b02905d11f3464c6ee"
server_key_iv: "78a1f993f7bf88510b0dfe0354cdd2b9"
server_key_version: "encryptionkey@0"
I20260807 08:48:27.924685 19884 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786092453486433-4734-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786092453486433-4734-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786092453486433-4734-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:48:27.927868 19884 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:48:27.928090 19884 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:48:27.928211 19884 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:48:27.928392 19884 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:48:27.928663 19884 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:48:27.928696 19884 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:48:27.928721 19884 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:48:27.928738 19884 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:48:27.933938 19884 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.131:46077
I20260807 08:48:27.933974 20097 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.131:46077 every 8 connection(s)
I20260807 08:48:27.934257 19884 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786092453486433-4734-0/minicluster-data/ts-2/data/info.pb
I20260807 08:48:27.935158 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 19884
I20260807 08:48:27.935241 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786092453486433-4734-0/minicluster-data/ts-2/wal/instance
I20260807 08:48:27.935416 4734 external_mini_cluster.cc:1428] Setting key 06ffce5199967f9a032ffb351e4eecc4
I20260807 08:48:27.937117 19428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:27.934926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.131:41637 (local address 127.4.159.190:45807)
0807 08:48:27.934986 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:27.934988 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:27.935241 (+ 253us) server_negotiation.cc:407] Connection header received
0807 08:48:27.935740 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:27.935741 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:27.935748 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:27.935789 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:27.936892 (+ 1103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:48:27.936894 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:48:27.936894 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:48:27.936898 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:48:27.936913 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:48:27.936917 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:48:27.936966 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0807 08:48:27.937036 (+ 70us) server_negotiation.cc:299] Negotiation successful
0807 08:48:27.937049 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260807 08:48:27.937304 20106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:27.934923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:45807 (local address 127.4.159.131:41637)
0807 08:48:27.935119 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:27.935156 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:27.935654 (+ 498us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:27.935853 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:27.935857 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:27.935866 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:48:27.936752 (+ 886us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 08:48:27.936753 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:48:27.936780 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 08:48:27.936807 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:48:27.936973 (+ 166us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:48:27.936977 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:48:27.936980 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 08:48:27.937046 (+ 66us) client_negotiation.cc:240] Negotiation successful
0807 08:48:27.937105 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":95,"threads_started":1}
I20260807 08:48:27.937711 20100 heartbeater.cc:347] Connected to a master server at 127.4.159.190:45807
I20260807 08:48:27.937791 20100 heartbeater.cc:464] Registering TS with master...
I20260807 08:48:27.937902 20100 heartbeater.cc:511] Master 127.4.159.190:45807 requested a full tablet report, sending...
I20260807 08:48:27.938251 19363 ts_manager.cc:194] Registered new tserver with Master: ad5cc22c6b8447288612d75325910ca2 (127.4.159.131:46077)
I20260807 08:48:27.947130 4734 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260807 08:48:27.948858 19428 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:48:27.948371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36074 (local address 127.4.159.190:45807)
0807 08:48:27.948424 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:27.948426 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:27.948531 (+ 105us) server_negotiation.cc:407] Connection header received
0807 08:48:27.948595 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:27.948595 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:27.948601 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:27.948649 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:27.948795 (+ 146us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36074: BlockingRecv error: recv got EOF from 127.0.0.1:36074 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
I20260807 08:48:27.948905 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 19461
I20260807 08:48:27.983794 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 19662
I20260807 08:48:27.992023 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 19884
I20260807 08:48:28.000293 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 19312
2026-08-07T08:48:28Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (579 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786092453486433-4734-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:48:28 dist-test-slave-6zbq krb5kdc[20116](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:48:28 dist-test-slave-6zbq krb5kdc[20116](info): set up 2 sockets
Aug 07 08:48:28 dist-test-slave-6zbq krb5kdc[20116](info): commencing operation
krb5kdc: starting...
W20260807 08:48:29.122331 4734 mini_kdc.cc:121] Time spent starting KDC: real 1.099s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786092453486433-4734-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786092453486433-4734-0/krb5kdc/test-user.keytab.
Aug 07 08:48:29 dist-test-slave-6zbq krb5kdc[20116](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092509, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:48:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:48:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.159.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.159.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.159.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab.
Entry for principal kudu/127.4.159.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab.
I20260807 08:48:29.229803 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786092453486433-4734-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786092453486433-4734-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786092453486433-4734-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.159.190:35179
--webserver_interface=127.4.159.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab
--principal=kudu/127.4.159.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:45411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.159.190:35179
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:48:29.318173 20137 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:48:29.318689 20137 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:48:29.319001 20137 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:48:29.320356 20137 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:48:29.320416 20137 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:48:29.320442 20137 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:48:29.320461 20137 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:48:29.320477 20137 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:48:29.322572 20137 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:45411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786092453486433-4734-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786092453486433-4734-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.159.190:35179
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab
--principal=kudu/127.4.159.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.159.190:35179
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.159.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20137
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786092453486433-4734-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:48:29.322944 20137 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:48:29.323184 20137 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:48:29.328383 20144 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:48:29.328459 20137 server_base.cc:1034] running on GCE node
W20260807 08:48:29.328388 20147 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:48:29.328382 20145 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:48:29.328967 20137 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:48:29.329275 20137 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:48:29.330406 20137 hybrid_clock.cc:648] HybridClock initialized: now 1786092509330388 us; error 28 us; skew 500 ppm
Aug 07 08:48:29 dist-test-slave-6zbq krb5kdc[20116](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092509, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:48:29.332818 20137 init.cc:377] Logged in from keytab as kudu/127.4.159.190@KRBTEST.COM (short username kudu)
I20260807 08:48:29.333262 20137 webserver.cc:460] Webserver started at http://127.4.159.190:42399/ using document root <none> and password file <none>
I20260807 08:48:29.333463 20137 fs_manager.cc:359] Metadata directory not provided
I20260807 08:48:29.333511 20137 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:48:29.333645 20137 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:48:29.334823 20137 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786092453486433-4734-0/minicluster-data/master-0/data/instance:
uuid: "37947189d3914053aed23acf9321b805"
format_stamp: "Formatted at 2026-08-07 08:48:29 on dist-test-slave-6zbq"
server_key: "fb1a27973a85a39805583c446051e8a1"
server_key_iv: "d927ce63a42a596c5bbeae3044e8c460"
server_key_version: "encryptionkey@0"
I20260807 08:48:29.335157 20137 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786092453486433-4734-0/minicluster-data/master-0/wal/instance:
uuid: "37947189d3914053aed23acf9321b805"
format_stamp: "Formatted at 2026-08-07 08:48:29 on dist-test-slave-6zbq"
server_key: "fb1a27973a85a39805583c446051e8a1"
server_key_iv: "d927ce63a42a596c5bbeae3044e8c460"
server_key_version: "encryptionkey@0"
I20260807 08:48:29.336377 20137 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 08:48:29.337037 20159 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:48:29.337203 20137 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 08:48:29.337291 20137 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786092453486433-4734-0/minicluster-data/master-0/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786092453486433-4734-0/minicluster-data/master-0/wal
uuid: "37947189d3914053aed23acf9321b805"
format_stamp: "Formatted at 2026-08-07 08:48:29 on dist-test-slave-6zbq"
server_key: "fb1a27973a85a39805583c446051e8a1"
server_key_iv: "d927ce63a42a596c5bbeae3044e8c460"
server_key_version: "encryptionkey@0"
I20260807 08:48:29.337349 20137 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786092453486433-4734-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786092453486433-4734-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786092453486433-4734-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:48:29.361569 20137 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:48:29.363751 20137 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:48:29.363891 20137 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:48:29.367969 20137 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.190:35179
I20260807 08:48:29.368016 20235 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.190:35179 every 8 connection(s)
I20260807 08:48:29.368341 20137 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
I20260807 08:48:29.369279 20238 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:48:29.371583 20238 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 37947189d3914053aed23acf9321b805: Bootstrap starting.
I20260807 08:48:29.372154 20238 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 37947189d3914053aed23acf9321b805: Neither blocks nor log segments found. Creating new log.
I20260807 08:48:29.372426 20238 log.cc:824] T 00000000000000000000000000000000 P 37947189d3914053aed23acf9321b805: Log is configured to *not* fsync() on all Append() calls
I20260807 08:48:29.373072 20238 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 37947189d3914053aed23acf9321b805: No bootstrap required, opened a new log
I20260807 08:48:29.374852 20238 raft_consensus.cc:348] T 00000000000000000000000000000000 P 37947189d3914053aed23acf9321b805 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37947189d3914053aed23acf9321b805" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 35179 } }
I20260807 08:48:29.374987 20238 raft_consensus.cc:374] T 00000000000000000000000000000000 P 37947189d3914053aed23acf9321b805 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:48:29.374989 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 20137
I20260807 08:48:29.375021 20238 raft_consensus.cc:729] T 00000000000000000000000000000000 P 37947189d3914053aed23acf9321b805 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37947189d3914053aed23acf9321b805, State: Initialized, Role: FOLLOWER
I20260807 08:48:29.375075 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786092453486433-4734-0/minicluster-data/master-0/wal/instance
I20260807 08:48:29.375173 20238 consensus_queue.cc:260] T 00000000000000000000000000000000 P 37947189d3914053aed23acf9321b805 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37947189d3914053aed23acf9321b805" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 35179 } }
I20260807 08:48:29.375253 20238 raft_consensus.cc:388] T 00000000000000000000000000000000 P 37947189d3914053aed23acf9321b805 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:48:29.375263 4734 external_mini_cluster.cc:1428] Setting key d1300dbd10af89b22f72166e4a7bc28b
I20260807 08:48:29.375286 20238 raft_consensus.cc:482] T 00000000000000000000000000000000 P 37947189d3914053aed23acf9321b805 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:48:29.375336 20238 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 37947189d3914053aed23acf9321b805 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:48:29.376330 20238 raft_consensus.cc:504] T 00000000000000000000000000000000 P 37947189d3914053aed23acf9321b805 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37947189d3914053aed23acf9321b805" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 35179 } }
I20260807 08:48:29.376438 20238 leader_election.cc:302] T 00000000000000000000000000000000 P 37947189d3914053aed23acf9321b805 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 37947189d3914053aed23acf9321b805; no voters:
I20260807 08:48:29.376581 20238 leader_election.cc:288] T 00000000000000000000000000000000 P 37947189d3914053aed23acf9321b805 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:48:29.376603 20246 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 37947189d3914053aed23acf9321b805 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:48:29.376709 20246 raft_consensus.cc:686] T 00000000000000000000000000000000 P 37947189d3914053aed23acf9321b805 [term 1 LEADER]: Becoming Leader. State: Replica: 37947189d3914053aed23acf9321b805, State: Running, Role: LEADER
I20260807 08:48:29.376843 20238 sys_catalog.cc:564] T 00000000000000000000000000000000 P 37947189d3914053aed23acf9321b805 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:48:29.376850 20246 consensus_queue.cc:237] T 00000000000000000000000000000000 P 37947189d3914053aed23acf9321b805 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37947189d3914053aed23acf9321b805" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 35179 } }
I20260807 08:48:29.377382 20248 sys_catalog.cc:455] T 00000000000000000000000000000000 P 37947189d3914053aed23acf9321b805 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 37947189d3914053aed23acf9321b805. Latest consensus state: current_term: 1 leader_uuid: "37947189d3914053aed23acf9321b805" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37947189d3914053aed23acf9321b805" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 35179 } } }
I20260807 08:48:29.377374 20251 sys_catalog.cc:455] T 00000000000000000000000000000000 P 37947189d3914053aed23acf9321b805 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "37947189d3914053aed23acf9321b805" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37947189d3914053aed23acf9321b805" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 35179 } } }
I20260807 08:48:29.377486 20248 sys_catalog.cc:458] T 00000000000000000000000000000000 P 37947189d3914053aed23acf9321b805 [sys.catalog]: This master's current role is: LEADER
I20260807 08:48:29.377493 20251 sys_catalog.cc:458] T 00000000000000000000000000000000 P 37947189d3914053aed23acf9321b805 [sys.catalog]: This master's current role is: LEADER
I20260807 08:48:29.378252 20260 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:48:29.378743 20260 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:48:29.381604 20260 catalog_manager.cc:1347] Generated new cluster ID: 1162d42d90b643a0a4d43a6c8fd11a03
I20260807 08:48:29.381659 20260 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 08:48:29 dist-test-slave-6zbq krb5kdc[20116](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092509, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.159.190@KRBTEST.COM
I20260807 08:48:29.385907 20245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:29.375748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57628 (local address 127.4.159.190:35179)
0807 08:48:29.375946 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:29.375958 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:29.375996 (+ 38us) server_negotiation.cc:407] Connection header received
0807 08:48:29.376132 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:29.376140 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:29.376198 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:29.376340 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:29.377007 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:29.378498 (+ 1491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:29.379249 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:29.379826 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:29.379933 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:29.382470 (+ 2537us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:48:29.382475 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:48:29.382475 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:48:29.382486 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:48:29.384344 (+ 1858us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:48:29.384734 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:48:29.384736 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:48:29.384737 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:48:29.384846 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:48:29.385045 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:48:29.385045 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:48:29.385046 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:48:29.385324 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:48:29.385459 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0807 08:48:29.385575 (+ 116us) server_negotiation.cc:299] Negotiation successful
0807 08:48:29.385703 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":90,"threads_started":1}
I20260807 08:48:29.389492 20260 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:48:29.390004 20260 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:48:29.392958 20260 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 37947189d3914053aed23acf9321b805: Generated new TSK 0
I20260807 08:48:29.393108 20260 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.159.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.159.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.159.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.129.keytab.
Entry for principal kudu/127.4.159.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.129.keytab.
I20260807 08:48:29.418303 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786092453486433-4734-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786092453486433-4734-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786092453486433-4734-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786092453486433-4734-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.159.129:0
--local_ip_for_outbound_sockets=127.4.159.129
--webserver_interface=127.4.159.129
--webserver_port=0
--tserver_master_addrs=127.4.159.190:35179
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.129.keytab
--principal=kudu/127.4.159.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:45411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:48:29.498831 20278 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:48:29.499007 20278 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:48:29.499038 20278 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:48:29.500128 20278 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:48:29.500167 20278 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:48:29.500201 20278 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.159.129
I20260807 08:48:29.501616 20278 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:45411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786092453486433-4734-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786092453486433-4734-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.129.keytab
--principal=kudu/127.4.159.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.159.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786092453486433-4734-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.159.129
--webserver_port=0
--tserver_master_addrs=127.4.159.190:35179
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20278
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.159.129
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786092453486433-4734-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:48:29.501845 20278 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:48:29.502046 20278 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:48:29.506641 20289 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:48:29.506670 20286 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:48:29.506651 20284 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:48:29.506847 20278 server_base.cc:1034] running on GCE node
I20260807 08:48:29.507073 20278 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:48:29.507268 20278 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:48:29.508399 20278 hybrid_clock.cc:648] HybridClock initialized: now 1786092509508388 us; error 25 us; skew 500 ppm
Aug 07 08:48:29 dist-test-slave-6zbq krb5kdc[20116](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092509, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:48:29.510841 20278 init.cc:377] Logged in from keytab as kudu/127.4.159.129@KRBTEST.COM (short username kudu)
I20260807 08:48:29.511207 20278 webserver.cc:460] Webserver started at http://127.4.159.129:40705/ using document root <none> and password file <none>
I20260807 08:48:29.511399 20278 fs_manager.cc:359] Metadata directory not provided
I20260807 08:48:29.511437 20278 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:48:29.511525 20278 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:48:29.512458 20278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786092453486433-4734-0/minicluster-data/ts-0/data/instance:
uuid: "b6dfd34cf77648d59a9dc32bff4d5c34"
format_stamp: "Formatted at 2026-08-07 08:48:29 on dist-test-slave-6zbq"
server_key: "145465e06bf46014fa3e3a2f3a79ea00"
server_key_iv: "c068352b87dcb6fca8aa3428ff291b45"
server_key_version: "encryptionkey@0"
I20260807 08:48:29.512732 20278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786092453486433-4734-0/minicluster-data/ts-0/wal/instance:
uuid: "b6dfd34cf77648d59a9dc32bff4d5c34"
format_stamp: "Formatted at 2026-08-07 08:48:29 on dist-test-slave-6zbq"
server_key: "145465e06bf46014fa3e3a2f3a79ea00"
server_key_iv: "c068352b87dcb6fca8aa3428ff291b45"
server_key_version: "encryptionkey@0"
I20260807 08:48:29.513875 20278 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 08:48:29.514501 20299 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:48:29.514652 20278 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 08:48:29.514734 20278 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786092453486433-4734-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786092453486433-4734-0/minicluster-data/ts-0/wal
uuid: "b6dfd34cf77648d59a9dc32bff4d5c34"
format_stamp: "Formatted at 2026-08-07 08:48:29 on dist-test-slave-6zbq"
server_key: "145465e06bf46014fa3e3a2f3a79ea00"
server_key_iv: "c068352b87dcb6fca8aa3428ff291b45"
server_key_version: "encryptionkey@0"
I20260807 08:48:29.514797 20278 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786092453486433-4734-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786092453486433-4734-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786092453486433-4734-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:48:29.530970 20278 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:48:29.532925 20278 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:48:29.533054 20278 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:48:29.533241 20278 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:48:29.533531 20278 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:48:29.533564 20278 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:48:29.533591 20278 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:48:29.533607 20278 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:48:29.539072 20278 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.129:44835
I20260807 08:48:29.539120 20489 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.129:44835 every 8 connection(s)
I20260807 08:48:29.539407 20278 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786092453486433-4734-0/minicluster-data/ts-0/data/info.pb
I20260807 08:48:29.542701 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 20278
I20260807 08:48:29.542788 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786092453486433-4734-0/minicluster-data/ts-0/wal/instance
I20260807 08:48:29.542956 4734 external_mini_cluster.cc:1428] Setting key 3e7e4fca41de4a3ed01410051053c02a
Aug 07 08:48:29 dist-test-slave-6zbq krb5kdc[20116](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092509, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.129@KRBTEST.COM for kudu/127.4.159.190@KRBTEST.COM
I20260807 08:48:29.547982 20245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:29.540058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.129:40773 (local address 127.4.159.190:35179)
0807 08:48:29.540113 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:29.540115 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:29.540284 (+ 169us) server_negotiation.cc:407] Connection header received
0807 08:48:29.540736 (+ 452us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:29.540737 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:29.540749 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:29.540795 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:29.541703 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:29.542193 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:29.543229 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:29.543460 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:29.543523 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:29.545471 (+ 1948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:48:29.545473 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:48:29.545473 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:48:29.545481 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:48:29.546719 (+ 1238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:48:29.547087 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:48:29.547088 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:48:29.547089 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:48:29.547141 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:48:29.547372 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:48:29.547372 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:48:29.547373 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:48:29.547481 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:48:29.547537 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 08:48:29.547796 (+ 259us) server_negotiation.cc:299] Negotiation successful
0807 08:48:29.547891 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260807 08:48:29.548108 20494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:29.540053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:35179 (local address 127.4.159.129:40773)
0807 08:48:29.540186 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:29.540219 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:29.540658 (+ 439us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:29.540861 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:29.540865 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:29.541452 (+ 587us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:48:29.541629 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:29.541634 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:29.542257 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:29.542258 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:29.543162 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:29.543165 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:29.543546 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:29.543548 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:29.543678 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:29.544087 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:48:29.544094 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:48:29.545378 (+ 1284us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:48:29.546797 (+ 1419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:48:29.546804 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:48:29.546806 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:48:29.547016 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:48:29.547200 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:48:29.547200 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:48:29.547201 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:48:29.547304 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:48:29.547542 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:48:29.547549 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:48:29.547675 (+ 126us) client_negotiation.cc:769] Sending connection context
0807 08:48:29.547759 (+ 84us) client_negotiation.cc:240] Negotiation successful
0807 08:48:29.547904 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":69,"threads_started":1}
I20260807 08:48:29.548552 20491 heartbeater.cc:347] Connected to a master server at 127.4.159.190:35179
I20260807 08:48:29.548653 20491 heartbeater.cc:464] Registering TS with master...
I20260807 08:48:29.548835 20491 heartbeater.cc:511] Master 127.4.159.190:35179 requested a full tablet report, sending...
I20260807 08:48:29.549340 20182 ts_manager.cc:194] Registered new tserver with Master: b6dfd34cf77648d59a9dc32bff4d5c34 (127.4.159.129:44835)
I20260807 08:48:29.549909 20182 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.159.129@KRBTEST.COM'} at 127.4.159.129:40773
WARNING: no policy specified for kudu/127.4.159.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.159.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.159.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.130.keytab.
Entry for principal kudu/127.4.159.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.130.keytab.
I20260807 08:48:29.567114 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786092453486433-4734-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786092453486433-4734-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786092453486433-4734-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786092453486433-4734-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.159.130:0
--local_ip_for_outbound_sockets=127.4.159.130
--webserver_interface=127.4.159.130
--webserver_port=0
--tserver_master_addrs=127.4.159.190:35179
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.130.keytab
--principal=kudu/127.4.159.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:45411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:48:29.646243 20497 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:48:29.646448 20497 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:48:29.646471 20497 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:48:29.647583 20497 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:48:29.647622 20497 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:48:29.647650 20497 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.159.130
I20260807 08:48:29.649063 20497 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:45411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786092453486433-4734-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786092453486433-4734-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.130.keytab
--principal=kudu/127.4.159.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.159.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786092453486433-4734-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.159.130
--webserver_port=0
--tserver_master_addrs=127.4.159.190:35179
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20497
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.159.130
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786092453486433-4734-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:48:29.649271 20497 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:48:29.649492 20497 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:48:29.654284 20506 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:48:29.654377 20497 server_base.cc:1034] running on GCE node
W20260807 08:48:29.654287 20510 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:48:29.654306 20508 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:48:29.654729 20497 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:48:29.654953 20497 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:48:29.656087 20497 hybrid_clock.cc:648] HybridClock initialized: now 1786092509656073 us; error 31 us; skew 500 ppm
Aug 07 08:48:29 dist-test-slave-6zbq krb5kdc[20116](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092509, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:48:29.658492 20497 init.cc:377] Logged in from keytab as kudu/127.4.159.130@KRBTEST.COM (short username kudu)
I20260807 08:48:29.658896 20497 webserver.cc:460] Webserver started at http://127.4.159.130:46321/ using document root <none> and password file <none>
I20260807 08:48:29.659093 20497 fs_manager.cc:359] Metadata directory not provided
I20260807 08:48:29.659150 20497 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:48:29.659281 20497 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:48:29.660336 20497 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786092453486433-4734-0/minicluster-data/ts-1/data/instance:
uuid: "41c7d8f3228b4383bd5208e55cc82a33"
format_stamp: "Formatted at 2026-08-07 08:48:29 on dist-test-slave-6zbq"
server_key: "42403a38582375387b6bf91e42ae6dd7"
server_key_iv: "953277e166110fa2d6c2ae27f1ab8294"
server_key_version: "encryptionkey@0"
I20260807 08:48:29.660681 20497 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786092453486433-4734-0/minicluster-data/ts-1/wal/instance:
uuid: "41c7d8f3228b4383bd5208e55cc82a33"
format_stamp: "Formatted at 2026-08-07 08:48:29 on dist-test-slave-6zbq"
server_key: "42403a38582375387b6bf91e42ae6dd7"
server_key_iv: "953277e166110fa2d6c2ae27f1ab8294"
server_key_version: "encryptionkey@0"
I20260807 08:48:29.661966 20497 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 08:48:29.662688 20519 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:48:29.662894 20497 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 08:48:29.662981 20497 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786092453486433-4734-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786092453486433-4734-0/minicluster-data/ts-1/wal
uuid: "41c7d8f3228b4383bd5208e55cc82a33"
format_stamp: "Formatted at 2026-08-07 08:48:29 on dist-test-slave-6zbq"
server_key: "42403a38582375387b6bf91e42ae6dd7"
server_key_iv: "953277e166110fa2d6c2ae27f1ab8294"
server_key_version: "encryptionkey@0"
I20260807 08:48:29.663043 20497 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786092453486433-4734-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786092453486433-4734-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786092453486433-4734-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:48:29.675804 20497 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:48:29.677572 20497 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:48:29.677722 20497 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:48:29.677932 20497 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:48:29.678241 20497 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:48:29.678277 20497 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:48:29.678296 20497 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:48:29.678315 20497 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:48:29.683753 20497 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.130:34853
I20260807 08:48:29.683805 20684 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.130:34853 every 8 connection(s)
I20260807 08:48:29.684161 20497 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786092453486433-4734-0/minicluster-data/ts-1/data/info.pb
Aug 07 08:48:29 dist-test-slave-6zbq krb5kdc[20116](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092509, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.130@KRBTEST.COM for kudu/127.4.159.190@KRBTEST.COM
I20260807 08:48:29.691421 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 20497
I20260807 08:48:29.691510 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786092453486433-4734-0/minicluster-data/ts-1/wal/instance
I20260807 08:48:29.691707 4734 external_mini_cluster.cc:1428] Setting key 686a101272095f125141d334688447fd
I20260807 08:48:29.692890 20245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:29.684878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.130:48605 (local address 127.4.159.190:35179)
0807 08:48:29.684941 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:29.684944 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:29.685105 (+ 161us) server_negotiation.cc:407] Connection header received
0807 08:48:29.685627 (+ 522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:29.685628 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:29.685640 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:29.685709 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:29.686668 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:29.687166 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:29.688070 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:29.688285 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:29.688336 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:29.690303 (+ 1967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:48:29.690304 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:48:29.690304 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:48:29.690311 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:48:29.691510 (+ 1199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:48:29.691919 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:48:29.691921 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:48:29.691921 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:48:29.691988 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:48:29.692222 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:48:29.692223 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:48:29.692224 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:48:29.692358 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:48:29.692427 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0807 08:48:29.692750 (+ 323us) server_negotiation.cc:299] Negotiation successful
0807 08:48:29.692824 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260807 08:48:29.693075 20687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:29.684823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:35179 (local address 127.4.159.130:48605)
0807 08:48:29.684986 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:29.685026 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:29.685508 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:29.685779 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:29.685783 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:29.686371 (+ 588us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:48:29.686558 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:29.686562 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:29.687238 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:29.687238 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:29.687994 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:29.687996 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:29.688347 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:29.688348 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:29.688437 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:29.688877 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:48:29.688884 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:48:29.690218 (+ 1334us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:48:29.691605 (+ 1387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:48:29.691612 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:48:29.691614 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:48:29.691844 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:48:29.692050 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:48:29.692050 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:48:29.692051 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:48:29.692158 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:48:29.692429 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:48:29.692436 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:48:29.692593 (+ 157us) client_negotiation.cc:769] Sending connection context
0807 08:48:29.692721 (+ 128us) client_negotiation.cc:240] Negotiation successful
0807 08:48:29.692867 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":86,"threads_started":1}
I20260807 08:48:29.693477 20685 heartbeater.cc:347] Connected to a master server at 127.4.159.190:35179
I20260807 08:48:29.693570 20685 heartbeater.cc:464] Registering TS with master...
I20260807 08:48:29.693742 20685 heartbeater.cc:511] Master 127.4.159.190:35179 requested a full tablet report, sending...
I20260807 08:48:29.694159 20182 ts_manager.cc:194] Registered new tserver with Master: 41c7d8f3228b4383bd5208e55cc82a33 (127.4.159.130:34853)
I20260807 08:48:29.694577 20182 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.159.130@KRBTEST.COM'} at 127.4.159.130:48605
WARNING: no policy specified for kudu/127.4.159.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.159.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.159.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.131.keytab.
Entry for principal kudu/127.4.159.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.131.keytab.
I20260807 08:48:29.714856 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786092453486433-4734-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786092453486433-4734-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786092453486433-4734-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786092453486433-4734-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.159.131:0
--local_ip_for_outbound_sockets=127.4.159.131
--webserver_interface=127.4.159.131
--webserver_port=0
--tserver_master_addrs=127.4.159.190:35179
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.131.keytab
--principal=kudu/127.4.159.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:45411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:48:29.793555 20694 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:48:29.793733 20694 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:48:29.793754 20694 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:48:29.794849 20694 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:48:29.794886 20694 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:48:29.794910 20694 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.159.131
I20260807 08:48:29.796609 20694 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:45411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786092453486433-4734-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786092453486433-4734-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.131.keytab
--principal=kudu/127.4.159.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.159.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786092453486433-4734-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.159.131
--webserver_port=0
--tserver_master_addrs=127.4.159.190:35179
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20694
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.159.131
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786092453486433-4734-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:48:29.796777 20694 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:48:29.796936 20694 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:48:29.801407 20705 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:48:29.801378 20701 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:48:29.801383 20700 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:48:29.801597 20694 server_base.cc:1034] running on GCE node
I20260807 08:48:29.801796 20694 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:48:29.802006 20694 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:48:29.803138 20694 hybrid_clock.cc:648] HybridClock initialized: now 1786092509803126 us; error 29 us; skew 500 ppm
Aug 07 08:48:29 dist-test-slave-6zbq krb5kdc[20116](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092509, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:48:29.805452 20694 init.cc:377] Logged in from keytab as kudu/127.4.159.131@KRBTEST.COM (short username kudu)
I20260807 08:48:29.805851 20694 webserver.cc:460] Webserver started at http://127.4.159.131:40731/ using document root <none> and password file <none>
I20260807 08:48:29.806032 20694 fs_manager.cc:359] Metadata directory not provided
I20260807 08:48:29.806077 20694 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:48:29.806198 20694 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:48:29.807163 20694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786092453486433-4734-0/minicluster-data/ts-2/data/instance:
uuid: "8f6a04b2ee3d40919283d7e44972d00c"
format_stamp: "Formatted at 2026-08-07 08:48:29 on dist-test-slave-6zbq"
server_key: "81a95161e15562598420ea905a62de2f"
server_key_iv: "44dbf79b4e909dcd92d415fac1833c8d"
server_key_version: "encryptionkey@0"
I20260807 08:48:29.807476 20694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786092453486433-4734-0/minicluster-data/ts-2/wal/instance:
uuid: "8f6a04b2ee3d40919283d7e44972d00c"
format_stamp: "Formatted at 2026-08-07 08:48:29 on dist-test-slave-6zbq"
server_key: "81a95161e15562598420ea905a62de2f"
server_key_iv: "44dbf79b4e909dcd92d415fac1833c8d"
server_key_version: "encryptionkey@0"
I20260807 08:48:29.808679 20694 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 08:48:29.809329 20719 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:48:29.809474 20694 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 08:48:29.809552 20694 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786092453486433-4734-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786092453486433-4734-0/minicluster-data/ts-2/wal
uuid: "8f6a04b2ee3d40919283d7e44972d00c"
format_stamp: "Formatted at 2026-08-07 08:48:29 on dist-test-slave-6zbq"
server_key: "81a95161e15562598420ea905a62de2f"
server_key_iv: "44dbf79b4e909dcd92d415fac1833c8d"
server_key_version: "encryptionkey@0"
I20260807 08:48:29.809628 20694 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786092453486433-4734-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786092453486433-4734-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786092453486433-4734-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:48:29.830971 20694 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:48:29.832876 20694 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:48:29.833003 20694 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:48:29.833191 20694 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:48:29.833453 20694 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:48:29.833487 20694 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:48:29.833513 20694 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:48:29.833529 20694 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:48:29.838740 20694 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.131:34111
I20260807 08:48:29.838800 20916 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.131:34111 every 8 connection(s)
I20260807 08:48:29.839049 20694 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786092453486433-4734-0/minicluster-data/ts-2/data/info.pb
I20260807 08:48:29.839210 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 20694
I20260807 08:48:29.839300 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786092453486433-4734-0/minicluster-data/ts-2/wal/instance
I20260807 08:48:29.839489 4734 external_mini_cluster.cc:1428] Setting key ab837b4bcb7f4873ae0ac0ba7048f405
Aug 07 08:48:29 dist-test-slave-6zbq krb5kdc[20116](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092509, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.131@KRBTEST.COM for kudu/127.4.159.190@KRBTEST.COM
I20260807 08:48:29.847091 20245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:29.839665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.131:57687 (local address 127.4.159.190:35179)
0807 08:48:29.839720 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:29.839722 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:29.839976 (+ 254us) server_negotiation.cc:407] Connection header received
0807 08:48:29.840482 (+ 506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:29.840483 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:29.840495 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:29.840544 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:29.841434 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:29.841923 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:29.842862 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:29.843066 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:29.843128 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:29.844967 (+ 1839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:48:29.844969 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:48:29.844969 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:48:29.844976 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:48:29.846026 (+ 1050us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:48:29.846406 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:48:29.846406 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:48:29.846407 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:48:29.846455 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:48:29.846646 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:48:29.846647 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:48:29.846647 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:48:29.846742 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:48:29.846787 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0807 08:48:29.846972 (+ 185us) server_negotiation.cc:299] Negotiation successful
0807 08:48:29.847029 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260807 08:48:29.847229 20920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:29.839638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:35179 (local address 127.4.159.131:57687)
0807 08:48:29.839862 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:29.839898 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:29.840397 (+ 499us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:29.840613 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:29.840617 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:29.841177 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:48:29.841341 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:29.841346 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:29.842005 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:29.842006 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:29.842776 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:29.842778 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:29.843134 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:29.843135 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:29.843257 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:29.843656 (+ 399us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:48:29.843663 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:48:29.844884 (+ 1221us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:48:29.846109 (+ 1225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:48:29.846116 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:48:29.846118 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:48:29.846312 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:48:29.846504 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:48:29.846504 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:48:29.846504 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:48:29.846595 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:48:29.846785 (+ 190us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:48:29.846787 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:48:29.846881 (+ 94us) client_negotiation.cc:769] Sending connection context
0807 08:48:29.846951 (+ 70us) client_negotiation.cc:240] Negotiation successful
0807 08:48:29.847054 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":83,"threads_started":1}
I20260807 08:48:29.847599 20918 heartbeater.cc:347] Connected to a master server at 127.4.159.190:35179
I20260807 08:48:29.847682 20918 heartbeater.cc:464] Registering TS with master...
I20260807 08:48:29.847831 20918 heartbeater.cc:511] Master 127.4.159.190:35179 requested a full tablet report, sending...
I20260807 08:48:29.848151 20182 ts_manager.cc:194] Registered new tserver with Master: 8f6a04b2ee3d40919283d7e44972d00c (127.4.159.131:34111)
I20260807 08:48:29.848484 20182 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.159.131@KRBTEST.COM'} at 127.4.159.131:57687
I20260807 08:48:29.850947 4734 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 08:48:29.857045 20245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:29.852081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57634 (local address 127.4.159.190:35179)
0807 08:48:29.852119 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:29.852121 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:29.852224 (+ 103us) server_negotiation.cc:407] Connection header received
0807 08:48:29.852260 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:29.852260 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:29.852268 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:29.852311 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:48:29.852783 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:29.853224 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:29.853957 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:29.854147 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:29.854210 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:29.855234 (+ 1024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:48:29.855235 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:48:29.855235 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:48:29.855241 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:48:29.856229 (+ 988us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:48:29.856530 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:48:29.856530 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:48:29.856531 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:48:29.856570 (+ 39us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:48:29.856686 (+ 116us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:48:29.856686 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:48:29.856686 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:48:29.856798 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:48:29.856863 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0807 08:48:29.856940 (+ 77us) server_negotiation.cc:299] Negotiation successful
0807 08:48:29.856990 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260807 08:48:29.858511 20182 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57634:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 08:48:29.859967 20182 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:48:29.865869 20932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:29.861847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.131:34111 (local address 127.0.0.1:38880)
0807 08:48:29.862296 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:29.862304 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:29.862363 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:29.862537 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:29.862540 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:29.862541 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:48:29.862620 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:29.862623 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:29.863538 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:29.863539 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:29.864896 (+ 1357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:29.864898 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:29.865569 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:29.865570 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:29.865658 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:29.865673 (+ 15us) client_negotiation.cc:769] Sending connection context
0807 08:48:29.865742 (+ 69us) client_negotiation.cc:240] Negotiation successful
0807 08:48:29.865795 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":439,"mutex_wait_us":91,"thread_start_us":46,"threads_started":1}
I20260807 08:48:29.866024 20935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:29.862091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35544 (local address 127.4.159.130:34853)
0807 08:48:29.862653 (+ 562us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:29.862657 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:29.862666 (+ 9us) server_negotiation.cc:407] Connection header received
0807 08:48:29.862721 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:29.862723 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:29.862738 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:29.862799 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:48:29.862903 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:29.863619 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:29.864940 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:29.865660 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:29.865730 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:29.865786 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 08:48:29.865901 (+ 115us) server_negotiation.cc:299] Negotiation successful
0807 08:48:29.865947 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":529,"thread_start_us":65,"threads_started":1}
I20260807 08:48:29.866148 20934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:29.861994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38880 (local address 127.4.159.131:34111)
0807 08:48:29.862396 (+ 402us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:29.862400 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:29.862409 (+ 9us) server_negotiation.cc:407] Connection header received
0807 08:48:29.862443 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:29.862445 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:29.862460 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:29.862524 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:48:29.862696 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:29.863444 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:29.864970 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:29.865490 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:29.865564 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:29.865625 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 08:48:29.866039 (+ 414us) server_negotiation.cc:299] Negotiation successful
0807 08:48:29.866077 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":69,"threads_started":1}
I20260807 08:48:29.866158 20930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:29.861855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.130:34853 (local address 127.0.0.1:35544)
0807 08:48:29.862430 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:29.862438 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:29.862472 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:29.862807 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:29.862808 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:29.862811 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:48:29.862854 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:29.862856 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:29.863726 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:29.863727 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:29.864826 (+ 1099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:29.864830 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:29.865742 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:29.865743 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:29.865817 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:29.865821 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 08:48:29.865876 (+ 55us) client_negotiation.cc:240] Negotiation successful
0807 08:48:29.866089 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":567,"mutex_wait_us":1,"thread_start_us":99,"threads_started":1}
I20260807 08:48:29.866699 20933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:29.861848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.129:44835 (local address 127.0.0.1:56868)
0807 08:48:29.862106 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:29.862144 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:29.862205 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:29.862537 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:29.862540 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:29.862543 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:48:29.862620 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:29.862625 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:29.864277 (+ 1652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:29.864278 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:29.865332 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:29.865335 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:29.866479 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:29.866480 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:29.866544 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:29.866548 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 08:48:29.866603 (+ 55us) client_negotiation.cc:240] Negotiation successful
0807 08:48:29.866636 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":229,"mutex_wait_us":1,"thread_start_us":46,"threads_started":1}
I20260807 08:48:29.866927 20936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:29.862175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56868 (local address 127.4.159.129:44835)
0807 08:48:29.862370 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:29.862375 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:29.862386 (+ 11us) server_negotiation.cc:407] Connection header received
0807 08:48:29.862429 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:29.862431 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:29.862445 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:29.862517 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:48:29.863540 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:29.864175 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:29.865411 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:29.866129 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:29.866460 (+ 331us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:29.866526 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 08:48:29.866806 (+ 280us) server_negotiation.cc:299] Negotiation successful
0807 08:48:29.866852 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":82,"threads_started":1}
I20260807 08:48:29.866832 20799 tablet_service.cc:1467] Processing CreateTablet for tablet ce8f0f291be9477a867f92a329e66003 (DEFAULT_TABLE table=test-table [id=9e2b2946db1d4c6cabef5a2e667321cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:48:29.866953 20585 tablet_service.cc:1467] Processing CreateTablet for tablet ce8f0f291be9477a867f92a329e66003 (DEFAULT_TABLE table=test-table [id=9e2b2946db1d4c6cabef5a2e667321cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:48:29.867164 20799 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce8f0f291be9477a867f92a329e66003. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:48:29.867300 20585 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce8f0f291be9477a867f92a329e66003. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:48:29.867566 20383 tablet_service.cc:1467] Processing CreateTablet for tablet ce8f0f291be9477a867f92a329e66003 (DEFAULT_TABLE table=test-table [id=9e2b2946db1d4c6cabef5a2e667321cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:48:29.867869 20383 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce8f0f291be9477a867f92a329e66003. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:48:29.869930 20937 tablet_bootstrap.cc:492] T ce8f0f291be9477a867f92a329e66003 P 41c7d8f3228b4383bd5208e55cc82a33: Bootstrap starting.
I20260807 08:48:29.870644 20937 tablet_bootstrap.cc:654] T ce8f0f291be9477a867f92a329e66003 P 41c7d8f3228b4383bd5208e55cc82a33: Neither blocks nor log segments found. Creating new log.
I20260807 08:48:29.871217 20941 tablet_bootstrap.cc:492] T ce8f0f291be9477a867f92a329e66003 P 8f6a04b2ee3d40919283d7e44972d00c: Bootstrap starting.
I20260807 08:48:29.871215 20937 log.cc:824] T ce8f0f291be9477a867f92a329e66003 P 41c7d8f3228b4383bd5208e55cc82a33: Log is configured to *not* fsync() on all Append() calls
I20260807 08:48:29.871591 20942 tablet_bootstrap.cc:492] T ce8f0f291be9477a867f92a329e66003 P b6dfd34cf77648d59a9dc32bff4d5c34: Bootstrap starting.
I20260807 08:48:29.871842 20941 tablet_bootstrap.cc:654] T ce8f0f291be9477a867f92a329e66003 P 8f6a04b2ee3d40919283d7e44972d00c: Neither blocks nor log segments found. Creating new log.
I20260807 08:48:29.871953 20937 tablet_bootstrap.cc:492] T ce8f0f291be9477a867f92a329e66003 P 41c7d8f3228b4383bd5208e55cc82a33: No bootstrap required, opened a new log
I20260807 08:48:29.872020 20937 ts_tablet_manager.cc:1397] T ce8f0f291be9477a867f92a329e66003 P 41c7d8f3228b4383bd5208e55cc82a33: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 08:48:29.872227 20942 tablet_bootstrap.cc:654] T ce8f0f291be9477a867f92a329e66003 P b6dfd34cf77648d59a9dc32bff4d5c34: Neither blocks nor log segments found. Creating new log.
I20260807 08:48:29.872411 20941 log.cc:824] T ce8f0f291be9477a867f92a329e66003 P 8f6a04b2ee3d40919283d7e44972d00c: Log is configured to *not* fsync() on all Append() calls
I20260807 08:48:29.872768 20942 log.cc:824] T ce8f0f291be9477a867f92a329e66003 P b6dfd34cf77648d59a9dc32bff4d5c34: Log is configured to *not* fsync() on all Append() calls
I20260807 08:48:29.873394 20942 tablet_bootstrap.cc:492] T ce8f0f291be9477a867f92a329e66003 P b6dfd34cf77648d59a9dc32bff4d5c34: No bootstrap required, opened a new log
I20260807 08:48:29.873445 20942 ts_tablet_manager.cc:1397] T ce8f0f291be9477a867f92a329e66003 P b6dfd34cf77648d59a9dc32bff4d5c34: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 08:48:29.873618 20937 raft_consensus.cc:348] T ce8f0f291be9477a867f92a329e66003 P 41c7d8f3228b4383bd5208e55cc82a33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f6a04b2ee3d40919283d7e44972d00c" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 34111 } } peers { permanent_uuid: "b6dfd34cf77648d59a9dc32bff4d5c34" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 44835 } } peers { permanent_uuid: "41c7d8f3228b4383bd5208e55cc82a33" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 34853 } }
I20260807 08:48:29.873759 20937 raft_consensus.cc:374] T ce8f0f291be9477a867f92a329e66003 P 41c7d8f3228b4383bd5208e55cc82a33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:48:29.873795 20937 raft_consensus.cc:729] T ce8f0f291be9477a867f92a329e66003 P 41c7d8f3228b4383bd5208e55cc82a33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41c7d8f3228b4383bd5208e55cc82a33, State: Initialized, Role: FOLLOWER
I20260807 08:48:29.873909 20937 consensus_queue.cc:260] T ce8f0f291be9477a867f92a329e66003 P 41c7d8f3228b4383bd5208e55cc82a33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f6a04b2ee3d40919283d7e44972d00c" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 34111 } } peers { permanent_uuid: "b6dfd34cf77648d59a9dc32bff4d5c34" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 44835 } } peers { permanent_uuid: "41c7d8f3228b4383bd5208e55cc82a33" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 34853 } }
I20260807 08:48:29.874167 20937 ts_tablet_manager.cc:1428] T ce8f0f291be9477a867f92a329e66003 P 41c7d8f3228b4383bd5208e55cc82a33: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 08:48:29.874229 20685 heartbeater.cc:503] Master 127.4.159.190:35179 was elected leader, sending a full tablet report...
I20260807 08:48:29.875032 20942 raft_consensus.cc:348] T ce8f0f291be9477a867f92a329e66003 P b6dfd34cf77648d59a9dc32bff4d5c34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f6a04b2ee3d40919283d7e44972d00c" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 34111 } } peers { permanent_uuid: "b6dfd34cf77648d59a9dc32bff4d5c34" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 44835 } } peers { permanent_uuid: "41c7d8f3228b4383bd5208e55cc82a33" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 34853 } }
I20260807 08:48:29.875169 20942 raft_consensus.cc:374] T ce8f0f291be9477a867f92a329e66003 P b6dfd34cf77648d59a9dc32bff4d5c34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:48:29.875214 20942 raft_consensus.cc:729] T ce8f0f291be9477a867f92a329e66003 P b6dfd34cf77648d59a9dc32bff4d5c34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6dfd34cf77648d59a9dc32bff4d5c34, State: Initialized, Role: FOLLOWER
I20260807 08:48:29.875319 20942 consensus_queue.cc:260] T ce8f0f291be9477a867f92a329e66003 P b6dfd34cf77648d59a9dc32bff4d5c34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f6a04b2ee3d40919283d7e44972d00c" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 34111 } } peers { permanent_uuid: "b6dfd34cf77648d59a9dc32bff4d5c34" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 44835 } } peers { permanent_uuid: "41c7d8f3228b4383bd5208e55cc82a33" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 34853 } }
I20260807 08:48:29.875550 20942 ts_tablet_manager.cc:1428] T ce8f0f291be9477a867f92a329e66003 P b6dfd34cf77648d59a9dc32bff4d5c34: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 08:48:29.875584 20491 heartbeater.cc:503] Master 127.4.159.190:35179 was elected leader, sending a full tablet report...
I20260807 08:48:29.875924 20941 tablet_bootstrap.cc:492] T ce8f0f291be9477a867f92a329e66003 P 8f6a04b2ee3d40919283d7e44972d00c: No bootstrap required, opened a new log
I20260807 08:48:29.875995 20941 ts_tablet_manager.cc:1397] T ce8f0f291be9477a867f92a329e66003 P 8f6a04b2ee3d40919283d7e44972d00c: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260807 08:48:29.877491 20941 raft_consensus.cc:348] T ce8f0f291be9477a867f92a329e66003 P 8f6a04b2ee3d40919283d7e44972d00c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f6a04b2ee3d40919283d7e44972d00c" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 34111 } } peers { permanent_uuid: "b6dfd34cf77648d59a9dc32bff4d5c34" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 44835 } } peers { permanent_uuid: "41c7d8f3228b4383bd5208e55cc82a33" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 34853 } }
I20260807 08:48:29.877605 20941 raft_consensus.cc:374] T ce8f0f291be9477a867f92a329e66003 P 8f6a04b2ee3d40919283d7e44972d00c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:48:29.877624 20941 raft_consensus.cc:729] T ce8f0f291be9477a867f92a329e66003 P 8f6a04b2ee3d40919283d7e44972d00c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f6a04b2ee3d40919283d7e44972d00c, State: Initialized, Role: FOLLOWER
I20260807 08:48:29.877700 20941 consensus_queue.cc:260] T ce8f0f291be9477a867f92a329e66003 P 8f6a04b2ee3d40919283d7e44972d00c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f6a04b2ee3d40919283d7e44972d00c" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 34111 } } peers { permanent_uuid: "b6dfd34cf77648d59a9dc32bff4d5c34" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 44835 } } peers { permanent_uuid: "41c7d8f3228b4383bd5208e55cc82a33" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 34853 } }
I20260807 08:48:29.877907 20941 ts_tablet_manager.cc:1428] T ce8f0f291be9477a867f92a329e66003 P 8f6a04b2ee3d40919283d7e44972d00c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 08:48:29.877990 20918 heartbeater.cc:503] Master 127.4.159.190:35179 was elected leader, sending a full tablet report...
I20260807 08:48:29.918555 20950 raft_consensus.cc:482] T ce8f0f291be9477a867f92a329e66003 P b6dfd34cf77648d59a9dc32bff4d5c34 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:48:29.918629 20950 raft_consensus.cc:504] T ce8f0f291be9477a867f92a329e66003 P b6dfd34cf77648d59a9dc32bff4d5c34 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f6a04b2ee3d40919283d7e44972d00c" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 34111 } } peers { permanent_uuid: "b6dfd34cf77648d59a9dc32bff4d5c34" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 44835 } } peers { permanent_uuid: "41c7d8f3228b4383bd5208e55cc82a33" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 34853 } }
I20260807 08:48:29.918947 20950 leader_election.cc:288] T ce8f0f291be9477a867f92a329e66003 P b6dfd34cf77648d59a9dc32bff4d5c34 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f6a04b2ee3d40919283d7e44972d00c (127.4.159.131:34111), 41c7d8f3228b4383bd5208e55cc82a33 (127.4.159.130:34853)
I20260807 08:48:29.922217 20935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:29.919163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.129:34503 (local address 127.4.159.130:34853)
0807 08:48:29.919208 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:29.919209 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:29.919340 (+ 131us) server_negotiation.cc:407] Connection header received
0807 08:48:29.919365 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:29.919366 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:29.919372 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:29.919413 (+ 41us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:48:29.919546 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:29.920215 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:29.921497 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:29.921887 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:29.921942 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:29.921948 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0807 08:48:29.922117 (+ 169us) server_negotiation.cc:299] Negotiation successful
0807 08:48:29.922156 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260807 08:48:29.922214 20953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:29.919088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.130:34853 (local address 127.4.159.129:34503)
0807 08:48:29.919270 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:29.919278 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:29.919329 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:29.919438 (+ 109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:29.919438 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:29.919440 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:48:29.919494 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:29.919495 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:29.920278 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:29.920279 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:29.921418 (+ 1139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:29.921421 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:29.921976 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:29.921977 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:29.922052 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:29.922055 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 08:48:29.922096 (+ 41us) client_negotiation.cc:240] Negotiation successful
0807 08:48:29.922132 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"mutex_wait_us":26,"thread_start_us":84,"threads_started":1}
I20260807 08:48:29.922286 20494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:29.919079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.131:34111 (local address 127.4.159.129:42859)
0807 08:48:29.919145 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:29.919155 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 08:48:29.919192 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:48:29.919310 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:48:29.919311 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:48:29.919313 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:48:29.919374 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:29.919377 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:29.920213 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:29.920214 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:29.921418 (+ 1204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:48:29.921421 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:48:29.922004 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:48:29.922005 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:48:29.922072 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:29.922075 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 08:48:29.922142 (+ 67us) client_negotiation.cc:240] Negotiation successful
0807 08:48:29.922179 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48,"mutex_wait_us":36}
I20260807 08:48:29.922257 20934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:29.919174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.129:42859 (local address 127.4.159.131:34111)
0807 08:48:29.919211 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:29.919213 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:29.919223 (+ 10us) server_negotiation.cc:407] Connection header received
0807 08:48:29.919245 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:29.919246 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:29.919254 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:29.919300 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:48:29.919434 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:29.920139 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:29.921530 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:29.921914 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:29.921983 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:29.921991 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0807 08:48:29.922161 (+ 170us) server_negotiation.cc:299] Negotiation successful
0807 08:48:29.922205 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 08:48:29.922595 20835 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce8f0f291be9477a867f92a329e66003" candidate_uuid: "b6dfd34cf77648d59a9dc32bff4d5c34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f6a04b2ee3d40919283d7e44972d00c" is_pre_election: true
I20260807 08:48:29.922636 20618 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce8f0f291be9477a867f92a329e66003" candidate_uuid: "b6dfd34cf77648d59a9dc32bff4d5c34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41c7d8f3228b4383bd5208e55cc82a33" is_pre_election: true
I20260807 08:48:29.922725 20835 raft_consensus.cc:2454] T ce8f0f291be9477a867f92a329e66003 P 8f6a04b2ee3d40919283d7e44972d00c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b6dfd34cf77648d59a9dc32bff4d5c34 in term 0.
I20260807 08:48:29.922742 20618 raft_consensus.cc:2454] T ce8f0f291be9477a867f92a329e66003 P 41c7d8f3228b4383bd5208e55cc82a33 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b6dfd34cf77648d59a9dc32bff4d5c34 in term 0.
I20260807 08:48:29.922928 20304 leader_election.cc:302] T ce8f0f291be9477a867f92a329e66003 P b6dfd34cf77648d59a9dc32bff4d5c34 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41c7d8f3228b4383bd5208e55cc82a33, b6dfd34cf77648d59a9dc32bff4d5c34; no voters:
I20260807 08:48:29.923071 20950 raft_consensus.cc:2781] T ce8f0f291be9477a867f92a329e66003 P b6dfd34cf77648d59a9dc32bff4d5c34 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:48:29.923115 20950 raft_consensus.cc:482] T ce8f0f291be9477a867f92a329e66003 P b6dfd34cf77648d59a9dc32bff4d5c34 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:48:29.923130 20950 raft_consensus.cc:3037] T ce8f0f291be9477a867f92a329e66003 P b6dfd34cf77648d59a9dc32bff4d5c34 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:48:29.923738 20950 raft_consensus.cc:504] T ce8f0f291be9477a867f92a329e66003 P b6dfd34cf77648d59a9dc32bff4d5c34 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f6a04b2ee3d40919283d7e44972d00c" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 34111 } } peers { permanent_uuid: "b6dfd34cf77648d59a9dc32bff4d5c34" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 44835 } } peers { permanent_uuid: "41c7d8f3228b4383bd5208e55cc82a33" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 34853 } }
I20260807 08:48:29.923851 20950 leader_election.cc:288] T ce8f0f291be9477a867f92a329e66003 P b6dfd34cf77648d59a9dc32bff4d5c34 [CANDIDATE]: Term 1 election: Requested vote from peers 8f6a04b2ee3d40919283d7e44972d00c (127.4.159.131:34111), 41c7d8f3228b4383bd5208e55cc82a33 (127.4.159.130:34853)
I20260807 08:48:29.923978 20618 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce8f0f291be9477a867f92a329e66003" candidate_uuid: "b6dfd34cf77648d59a9dc32bff4d5c34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41c7d8f3228b4383bd5208e55cc82a33"
I20260807 08:48:29.924052 20618 raft_consensus.cc:3037] T ce8f0f291be9477a867f92a329e66003 P 41c7d8f3228b4383bd5208e55cc82a33 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:48:29.924062 20835 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce8f0f291be9477a867f92a329e66003" candidate_uuid: "b6dfd34cf77648d59a9dc32bff4d5c34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f6a04b2ee3d40919283d7e44972d00c"
I20260807 08:48:29.924139 20835 raft_consensus.cc:3037] T ce8f0f291be9477a867f92a329e66003 P 8f6a04b2ee3d40919283d7e44972d00c [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:48:29.924688 20618 raft_consensus.cc:2454] T ce8f0f291be9477a867f92a329e66003 P 41c7d8f3228b4383bd5208e55cc82a33 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b6dfd34cf77648d59a9dc32bff4d5c34 in term 1.
I20260807 08:48:29.924757 20835 raft_consensus.cc:2454] T ce8f0f291be9477a867f92a329e66003 P 8f6a04b2ee3d40919283d7e44972d00c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b6dfd34cf77648d59a9dc32bff4d5c34 in term 1.
I20260807 08:48:29.924839 20304 leader_election.cc:302] T ce8f0f291be9477a867f92a329e66003 P b6dfd34cf77648d59a9dc32bff4d5c34 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41c7d8f3228b4383bd5208e55cc82a33, b6dfd34cf77648d59a9dc32bff4d5c34; no voters:
I20260807 08:48:29.924925 20950 raft_consensus.cc:2781] T ce8f0f291be9477a867f92a329e66003 P b6dfd34cf77648d59a9dc32bff4d5c34 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:48:29.925041 20950 raft_consensus.cc:686] T ce8f0f291be9477a867f92a329e66003 P b6dfd34cf77648d59a9dc32bff4d5c34 [term 1 LEADER]: Becoming Leader. State: Replica: b6dfd34cf77648d59a9dc32bff4d5c34, State: Running, Role: LEADER
I20260807 08:48:29.925143 20950 consensus_queue.cc:237] T ce8f0f291be9477a867f92a329e66003 P b6dfd34cf77648d59a9dc32bff4d5c34 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f6a04b2ee3d40919283d7e44972d00c" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 34111 } } peers { permanent_uuid: "b6dfd34cf77648d59a9dc32bff4d5c34" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 44835 } } peers { permanent_uuid: "41c7d8f3228b4383bd5208e55cc82a33" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 34853 } }
I20260807 08:48:29.925804 20181 catalog_manager.cc:5495] T ce8f0f291be9477a867f92a329e66003 P b6dfd34cf77648d59a9dc32bff4d5c34 reported cstate change: term changed from 0 to 1, leader changed from <none> to b6dfd34cf77648d59a9dc32bff4d5c34 (127.4.159.129). New cstate: current_term: 1 leader_uuid: "b6dfd34cf77648d59a9dc32bff4d5c34" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f6a04b2ee3d40919283d7e44972d00c" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 34111 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b6dfd34cf77648d59a9dc32bff4d5c34" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 44835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41c7d8f3228b4383bd5208e55cc82a33" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 34853 } health_report { overall_health: UNKNOWN } } }
W20260807 08:48:29.934629 20686 tablet.cc:2369] T ce8f0f291be9477a867f92a329e66003 P 41c7d8f3228b4383bd5208e55cc82a33: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 08:48:29.939921 20936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:29.937869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56880 (local address 127.4.159.129:44835)
0807 08:48:29.937907 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0807 08:48:29.937908 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 08:48:29.937916 (+ 8us) server_negotiation.cc:407] Connection header received
0807 08:48:29.937975 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:48:29.937975 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:48:29.937989 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:48:29.938034 (+ 45us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:48:29.938131 (+ 97us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:29.938605 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:29.939269 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:48:29.939453 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:48:29.939492 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:48:29.939640 (+ 148us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:48:29.939727 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:48:29.939783 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 08:48:29.939828 (+ 45us) server_negotiation.cc:299] Negotiation successful
0807 08:48:29.939843 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 08:48:29.941432 20835 raft_consensus.cc:1261] T ce8f0f291be9477a867f92a329e66003 P 8f6a04b2ee3d40919283d7e44972d00c [term 1 FOLLOWER]: Refusing update from remote peer b6dfd34cf77648d59a9dc32bff4d5c34: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:48:29.941442 20618 raft_consensus.cc:1261] T ce8f0f291be9477a867f92a329e66003 P 41c7d8f3228b4383bd5208e55cc82a33 [term 1 FOLLOWER]: Refusing update from remote peer b6dfd34cf77648d59a9dc32bff4d5c34: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:48:29.941748 20954 consensus_queue.cc:1035] T ce8f0f291be9477a867f92a329e66003 P b6dfd34cf77648d59a9dc32bff4d5c34 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41c7d8f3228b4383bd5208e55cc82a33" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 34853 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:48:29.941866 20950 consensus_queue.cc:1035] T ce8f0f291be9477a867f92a329e66003 P b6dfd34cf77648d59a9dc32bff4d5c34 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f6a04b2ee3d40919283d7e44972d00c" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 34111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:48:29.943781 20958 mvcc.cc:205] Tried to move back new op lower bound from 7315834920717803520 to 7315834920653897728. Current Snapshot: MvccSnapshot[applied={T|T < 7315834920717803520}]
I20260807 08:48:29.944114 20959 mvcc.cc:205] Tried to move back new op lower bound from 7315834920717803520 to 7315834920653897728. Current Snapshot: MvccSnapshot[applied={T|T < 7315834920717803520}]
I20260807 08:48:29.945086 20960 mvcc.cc:205] Tried to move back new op lower bound from 7315834920717803520 to 7315834920653897728. Current Snapshot: MvccSnapshot[applied={T|T < 7315834920717803520 or (T in {7315834920717803520})}]
I20260807 08:48:29.946136 20182 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57634:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:48:29.946234 20182 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57634:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:48:29.946986 20182 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 37947189d3914053aed23acf9321b805: Sending DeleteTablet for 3 replicas of tablet ce8f0f291be9477a867f92a329e66003
I20260807 08:48:29.947309 20585 tablet_service.cc:1514] Processing DeleteTablet for tablet ce8f0f291be9477a867f92a329e66003 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:48:29 UTC) from {username='kudu', principal='kudu/127.4.159.190@KRBTEST.COM'} at 127.0.0.1:35544
I20260807 08:48:29.947304 20799 tablet_service.cc:1514] Processing DeleteTablet for tablet ce8f0f291be9477a867f92a329e66003 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:48:29 UTC) from {username='kudu', principal='kudu/127.4.159.190@KRBTEST.COM'} at 127.0.0.1:38880
I20260807 08:48:29.947347 20383 tablet_service.cc:1514] Processing DeleteTablet for tablet ce8f0f291be9477a867f92a329e66003 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:48:29 UTC) from {username='kudu', principal='kudu/127.4.159.190@KRBTEST.COM'} at 127.0.0.1:56868
I20260807 08:48:29.947547 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 20278
I20260807 08:48:29.965778 20972 tablet_replica.cc:330] T ce8f0f291be9477a867f92a329e66003 P 41c7d8f3228b4383bd5208e55cc82a33: stopping tablet replica
I20260807 08:48:29.965884 20972 raft_consensus.cc:2229] T ce8f0f291be9477a867f92a329e66003 P 41c7d8f3228b4383bd5208e55cc82a33 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:48:29.965971 20972 raft_consensus.cc:2258] T ce8f0f291be9477a867f92a329e66003 P 41c7d8f3228b4383bd5208e55cc82a33 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:48:29.966738 20972 ts_tablet_manager.cc:1905] T ce8f0f291be9477a867f92a329e66003 P 41c7d8f3228b4383bd5208e55cc82a33: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:48:29.968314 20972 ts_tablet_manager.cc:1918] T ce8f0f291be9477a867f92a329e66003 P 41c7d8f3228b4383bd5208e55cc82a33: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:48:29.968391 20972 log.cc:1196] T ce8f0f291be9477a867f92a329e66003 P 41c7d8f3228b4383bd5208e55cc82a33: Deleting WAL directory at /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786092453486433-4734-0/minicluster-data/ts-1/wal/wals/ce8f0f291be9477a867f92a329e66003
I20260807 08:48:29.968631 20972 ts_tablet_manager.cc:1939] T ce8f0f291be9477a867f92a329e66003 P 41c7d8f3228b4383bd5208e55cc82a33: Deleting consensus metadata
I20260807 08:48:29.969029 20975 tablet_replica.cc:330] T ce8f0f291be9477a867f92a329e66003 P 8f6a04b2ee3d40919283d7e44972d00c: stopping tablet replica
I20260807 08:48:29.969028 20163 catalog_manager.cc:4841] TS 41c7d8f3228b4383bd5208e55cc82a33 (127.4.159.130:34853): tablet ce8f0f291be9477a867f92a329e66003 (table test-table [id=9e2b2946db1d4c6cabef5a2e667321cf]) successfully deleted
I20260807 08:48:29.969141 20975 raft_consensus.cc:2229] T ce8f0f291be9477a867f92a329e66003 P 8f6a04b2ee3d40919283d7e44972d00c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:48:29.969254 20975 raft_consensus.cc:2258] T ce8f0f291be9477a867f92a329e66003 P 8f6a04b2ee3d40919283d7e44972d00c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:48:29.969627 20975 ts_tablet_manager.cc:1905] T ce8f0f291be9477a867f92a329e66003 P 8f6a04b2ee3d40919283d7e44972d00c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:48:29.971072 20975 ts_tablet_manager.cc:1918] T ce8f0f291be9477a867f92a329e66003 P 8f6a04b2ee3d40919283d7e44972d00c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:48:29.971143 20975 log.cc:1196] T ce8f0f291be9477a867f92a329e66003 P 8f6a04b2ee3d40919283d7e44972d00c: Deleting WAL directory at /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786092453486433-4734-0/minicluster-data/ts-2/wal/wals/ce8f0f291be9477a867f92a329e66003
I20260807 08:48:29.971387 20975 ts_tablet_manager.cc:1939] T ce8f0f291be9477a867f92a329e66003 P 8f6a04b2ee3d40919283d7e44972d00c: Deleting consensus metadata
I20260807 08:48:29.971802 20165 catalog_manager.cc:4841] TS 8f6a04b2ee3d40919283d7e44972d00c (127.4.159.131:34111): tablet ce8f0f291be9477a867f92a329e66003 (table test-table [id=9e2b2946db1d4c6cabef5a2e667321cf]) successfully deleted
W20260807 08:48:29.972424 20161 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.159.129:44835 (error 108)
I20260807 08:48:29.972865 20933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:48:29.972744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.129:44835 (local address 127.0.0.1:56896)
0807 08:48:29.972793 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0807 08:48:29.972832 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.129:44835: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260807 08:48:29.972892 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 20497
W20260807 08:48:29.972997 20161 catalog_manager.cc:4584] TS b6dfd34cf77648d59a9dc32bff4d5c34 (127.4.159.129:44835): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ce8f0f291be9477a867f92a329e66003: Network error: Client connection negotiation failed: client connection to 127.4.159.129:44835: connect: Connection refused (error 111)
I20260807 08:48:29.987103 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 20694
I20260807 08:48:29.994478 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 20137
2026-08-07T08:48:29Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (1993 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:48:30 dist-test-slave-6zbq krb5kdc[20982](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:48:30 dist-test-slave-6zbq krb5kdc[20982](info): set up 2 sockets
Aug 07 08:48:30 dist-test-slave-6zbq krb5kdc[20982](info): commencing operation
krb5kdc: starting...
W20260807 08:48:31.105849 4734 mini_kdc.cc:121] Time spent starting KDC: real 1.089s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/krb5kdc/test-user.keytab.
Aug 07 08:48:31 dist-test-slave-6zbq krb5kdc[20982](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092511, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:48:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:48:31Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.4.159.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.4.159.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.4.159.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/krb5kdc/rangeradmin_127.4.159.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.4.159.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/krb5kdc/rangeradmin_127.4.159.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.4.159.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.4.159.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.4.159.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/krb5kdc/rangerlookup_127.4.159.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.4.159.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/krb5kdc/rangerlookup_127.4.159.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.4.159.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.4.159.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.4.159.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/krb5kdc/HTTP_127.4.159.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.4.159.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/krb5kdc/HTTP_127.4.159.148@KRBTEST.COM.keytab.
I20260807 08:48:31.254820 4734 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskQz81E_/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/minicluster-data/postgres -l logfile start
2026-08-07 08:48:33.452 UTC [21029] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-07 08:48:33.452 UTC [21029] LOG: listening on IPv4 address "127.4.159.148", port 50557
2026-08-07 08:48:33.455 UTC [21029] LOG: listening on Unix socket "/tmp/.s.PGSQL.50557"
2026-08-07 08:48:33.475 UTC [21032] LOG: database system was shut down at 2026-08-07 08:48:32 UTC
2026-08-07 08:48:33.478 UTC [21029] LOG: database system is ready to accept connections
I20260807 08:48:34.511137 4734 mini_postgres.cc:96] Postgres bound to 50557
2026-08-07 08:48:34.517 UTC [21042] FATAL: database "slave" does not exist
127.4.159.148:50557 - accepting connections
I20260807 08:48:34.518465 4734 mini_ranger.cc:161] Starting Ranger...
I20260807 08:48:34.532191 4734 mini_ranger.cc:84] Created miniranger Postgres user
I20260807 08:48:34.908150 4734 mini_ranger.cc:87] Created ranger Postgres database
I20260807 08:48:34.908236 4734 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/minicluster-data/ranger-admin
2026-08-07 08:48:35,185 [I] DB FLAVOR :POSTGRES
2026-08-07 08:48:35,186 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 08:48:35,186 [I] Checking connection..
2026-08-07 08:48:35,186 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQz81E_/build/release/bin/postgresql.jar:/tmp/dist-test-taskQz81E_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.159.148:50557/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-07 08:48:35,428 [I] Checking connection passed.
2026-08-07 08:48:35,428 [I] --------- Verifying version history table ---------
2026-08-07 08:48:35,428 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQz81E_/build/release/bin/postgresql.jar:/tmp/dist-test-taskQz81E_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.159.148:50557/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 08:48:35,656 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 08:48:35,657 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQz81E_/build/release/bin/postgresql.jar:/tmp/dist-test-taskQz81E_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.159.148:50557/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 08:48:35,908 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 08:48:35,909 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-07 08:48:35,909 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQz81E_/build/release/bin/postgresql.jar:/tmp/dist-test-taskQz81E_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.159.148:50557/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskQz81E_/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-07 08:48:36.144 UTC [21134] WARNING: there is no transaction in progress
2026-08-07 08:48:36,151 [I] create_dbversion_catalog.sql file imported successfully
2026-08-07 08:48:36,152 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQz81E_/build/release/bin/postgresql.jar:/tmp/dist-test-taskQz81E_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.159.148:50557/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 08:48:36,386 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-07 08:48:36,387 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 08:48:36,387 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQz81E_/build/release/bin/postgresql.jar:/tmp/dist-test-taskQz81E_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.159.148:50557/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-07 08:48:36,629 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQz81E_/build/release/bin/postgresql.jar:/tmp/dist-test-taskQz81E_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.159.148:50557/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-07 08:48:36,875 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQz81E_/build/release/bin/postgresql.jar:/tmp/dist-test-taskQz81E_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.159.148:50557/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-6zbq.c.gcp-upstream.internal','N') ;"
2026-08-07 08:48:37,119 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQz81E_/build/release/bin/postgresql.jar:/tmp/dist-test-taskQz81E_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.159.148:50557/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 08:48:37,340 [I] Table x_portal_user does not exist in database ranger
2026-08-07 08:48:37,341 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQz81E_/build/release/bin/postgresql.jar:/tmp/dist-test-taskQz81E_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.159.148:50557/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 08:48:37,584 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-07 08:48:37,585 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQz81E_/build/release/bin/postgresql.jar:/tmp/dist-test-taskQz81E_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.159.148:50557/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 08:48:37,837 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQz81E_/build/release/bin/postgresql.jar:/tmp/dist-test-taskQz81E_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.159.148:50557/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 08:48:38,084 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-07 08:48:38,085 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQz81E_/build/release/bin/postgresql.jar:/tmp/dist-test-taskQz81E_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.159.148:50557/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskQz81E_/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-07 08:48:38.972 UTC [21290] WARNING: there is no transaction in progress
2026-08-07 08:48:38.986 UTC [21290] WARNING: there is no transaction in progress
2026-08-07 08:48:39.000 UTC [21290] WARNING: there is no transaction in progress
2026-08-07 08:48:39.014 UTC [21290] WARNING: there is no transaction in progress
2026-08-07 08:48:39.029 UTC [21290] WARNING: there is no transaction in progress
2026-08-07 08:48:39.043 UTC [21290] WARNING: there is no transaction in progress
2026-08-07 08:48:39.091 UTC [21290] WARNING: there is no transaction in progress
2026-08-07 08:48:39.097 UTC [21290] WARNING: there is no transaction in progress
2026-08-07 08:48:39.109 UTC [21290] WARNING: there is no transaction in progress
2026-08-07 08:48:39.119 UTC [21290] WARNING: there is no transaction in progress
2026-08-07 08:48:39.128 UTC [21290] WARNING: there is no transaction in progress
2026-08-07 08:48:39.142 UTC [21290] WARNING: there is no transaction in progress
2026-08-07 08:48:39.151 UTC [21290] WARNING: there is no transaction in progress
2026-08-07 08:48:39.157 UTC [21290] WARNING: there is no transaction in progress
2026-08-07 08:48:39.162 UTC [21290] WARNING: there is no transaction in progress
2026-08-07 08:48:39.167 UTC [21290] WARNING: there is no transaction in progress
2026-08-07 08:48:39,696 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-07 08:48:39,696 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQz81E_/build/release/bin/postgresql.jar:/tmp/dist-test-taskQz81E_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.159.148:50557/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-07 08:48:39,940 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-07 08:48:39,940 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQz81E_/build/release/bin/postgresql.jar:/tmp/dist-test-taskQz81E_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.159.148:50557/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 08:48:40,174 [I] Table x_portal_user already exists in database 'ranger'
2026-08-07 08:48:40,175 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQz81E_/build/release/bin/postgresql.jar:/tmp/dist-test-taskQz81E_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.159.148:50557/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 08:48:40,427 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-07 08:48:40,428 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQz81E_/build/release/bin/postgresql.jar:/tmp/dist-test-taskQz81E_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.159.148:50557/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 08:48:40,674 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQz81E_/build/release/bin/postgresql.jar:/tmp/dist-test-taskQz81E_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.159.148:50557/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 08:48:40,914 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQz81E_/build/release/bin/postgresql.jar:/tmp/dist-test-taskQz81E_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.159.148:50557/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-6zbq.c.gcp-upstream.internal';"
2026-08-07 08:48:41,157 [I] CORE_DB_SCHEMA import status has been updated
2026-08-07 08:48:41,158 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQz81E_/build/release/bin/postgresql.jar:/tmp/dist-test-taskQz81E_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.159.148:50557/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-07 08:48:41,393 [I] DB_PATCHES have already been applied
I20260807 08:48:41.398458 4734 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/minicluster-data/ranger-admin:/tmp/dist-test-taskQz81E_/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskQz81E_/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskQz81E_/build/release/bin/postgresql.jar:/tmp/dist-test-taskQz81E_/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260807 08:48:41.398512 4734 mini_ranger.cc:193] Using host: 127.4.159.148
I20260807 08:48:41.402240 4734 mini_ranger.cc:227] Ranger admin URL: http://127.4.159.148:50679
Aug 07, 2026 8:48:41 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 8:48:41 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 8:48:41 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/minicluster-data/ranger-admin/ews/webapp
Aug 07, 2026 8:48:41 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 07, 2026 8:48:41 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 07, 2026 8:48:42 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/minicluster-data/ranger-admin/ews/webapp].
Aug 07, 2026 8:48:42 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260807 08:48:42.625811 4734 mini_ranger.cc:160] Time spent starting Ranger: real 8.107s user 0.006s sys 0.003s
Aug 07, 2026 8:49:02 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 07, 2026 8:49:02 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 07, 2026 8:49:02 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 07, 2026 8:49:02 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 07, 2026 8:49:02 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 07, 2026 8:49:02 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 07, 2026 8:49:02 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 07, 2026 8:49:02 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 07, 2026 8:49:02 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 07, 2026 8:49:02 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 07, 2026 8:49:02 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 07, 2026 8:49:02 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 07, 2026 8:49:02 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 07, 2026 8:49:02 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 07, 2026 8:49:02 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 07, 2026 8:49:02 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 07, 2026 8:49:02 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 07, 2026 8:49:03 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260807 08:49:03.786921 4734 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.4.159.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.4.159.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.4.159.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/krb5kdc/rangerkms_127.4.159.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.4.159.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/krb5kdc/rangerkms_127.4.159.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.4.159.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.4.159.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.4.159.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/krb5kdc/HTTP_127.4.159.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.4.159.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/krb5kdc/HTTP_127.4.159.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 07 08:49:03 dist-test-slave-6zbq krb5kdc[20982](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092543, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260807 08:49:03.856634 4734 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260807 08:49:03.864877 4734 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260807 08:49:04.502701 4734 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260807 08:49:04.502781 4734 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/minicluster-data/ranger-kms
I20260807 08:49:04.502796 4734 mini_ranger_kms.cc:227] Using postgres at 127.4.159.148:50557
2026-08-07 08:49:04,737 [I] DB FLAVOR :POSTGRES
2026-08-07 08:49:04,737 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 08:49:04,737 [I] Checking connection
2026-08-07 08:49:04,737 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQz81E_/build/release/bin/postgresql.jar:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.159.148:50557/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-07 08:49:05,020 [I] connection success
2026-08-07 08:49:05,020 [I] --------- Verifying Ranger DB tables ---------
2026-08-07 08:49:05,020 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-07 08:49:05,020 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQz81E_/build/release/bin/postgresql.jar:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.159.148:50557/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-07 08:49:05,245 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-07 08:49:05,246 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 08:49:05,246 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-07 08:49:05,246 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQz81E_/build/release/bin/postgresql.jar:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.159.148:50557/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-07 08:49:05,485 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260807 08:49:05.910009 4734 mini_ranger_kms.cc:310] Created kms service
I20260807 08:49:06.113348 4734 mini_ranger_kms.cc:326] Created kudu user
I20260807 08:49:06.152684 4734 mini_ranger_kms.cc:343] Created rangerkms user
Aug 07 08:49:06 dist-test-slave-6zbq krb5kdc[20982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092543, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.4.159.148@KRBTEST.COM
I20260807 08:49:06.568301 4734 mini_ranger_kms.cc:384] Added ranger policy
I20260807 08:49:06.568416 4734 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/minicluster-data/ranger-kms:/tmp/dist-test-taskQz81E_/build/release/bin/postgresql.jar:/tmp/dist-test-taskQz81E_/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskQz81E_/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260807 08:49:06.568449 4734 mini_ranger_kms.cc:240] Using host: 127.4.159.148
I20260807 08:49:06.572467 4734 mini_ranger_kms.cc:276] Ranger KMS PID: 21558
I20260807 08:49:06.572582 4734 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.4.159.148:40875
Aug 07, 2026 8:49:07 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 8:49:07 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 8:49:07 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 07, 2026 8:49:07 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 07, 2026 8:49:08 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/minicluster-data/ranger-kms/ews/webapp].
Aug 07, 2026 8:49:08 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 07 08:49:08 dist-test-slave-6zbq krb5kdc[20982](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786092548, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.4.159.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:49:09.027549 4734 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.171s user 0.009s sys 0.201s
I20260807 08:49:09.027683 4734 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260807 08:49:09.027781 4734 mini_ranger_kms.cc:401] 127.4.159.148:40875/kms/v1/keys
Aug 07 08:49:09 dist-test-slave-6zbq krb5kdc[20982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092548, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.4.159.148@KRBTEST.COM for HTTP/127.4.159.148@KRBTEST.COM
Aug 07 08:49:09 dist-test-slave-6zbq krb5kdc[20982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092548, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.4.159.148@KRBTEST.COM for HTTP/127.4.159.148@KRBTEST.COM
Aug 07 08:49:12 dist-test-slave-6zbq krb5kdc[20982](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092552, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.4.159.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.159.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.159.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab.
Entry for principal kudu/127.4.159.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab.
I20260807 08:49:12.146478 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.159.190:41811
--webserver_interface=127.4.159.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab
--principal=kudu/127.4.159.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:43823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.159.190:41811
--ranger_config_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.159.148:40875/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:49:12.251204 21630 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:49:12.251453 21630 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:49:12.251480 21630 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:49:12.253197 21630 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:49:12.253243 21630 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:49:12.253257 21630 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:49:12.253268 21630 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:49:12.253279 21630 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:49:12.255177 21630 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:43823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/minicluster-data/master-0/wal
--ranger_kms_url=127.4.159.148:40875/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.4.159.190:41811
--ranger_config_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab
--principal=kudu/127.4.159.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.159.190:41811
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.159.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21630
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:49:12.255388 21630 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:49:12.255576 21630 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:49:12.261324 21648 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:49:12.261322 21640 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:49:12.261319 21639 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:49:12.261351 21630 server_base.cc:1034] running on GCE node
I20260807 08:49:12.261718 21630 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:49:12.262004 21630 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:49:12.263180 21630 hybrid_clock.cc:648] HybridClock initialized: now 1786092552263160 us; error 28 us; skew 500 ppm
Aug 07 08:49:12 dist-test-slave-6zbq krb5kdc[20982](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092552, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:49:12.265592 21630 init.cc:377] Logged in from keytab as kudu/127.4.159.190@KRBTEST.COM (short username kudu)
I20260807 08:49:12.266019 21630 webserver.cc:460] Webserver started at http://127.4.159.190:37235/ using document root <none> and password file <none>
I20260807 08:49:12.266232 21630 fs_manager.cc:359] Metadata directory not provided
I20260807 08:49:12.266273 21630 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:49:12.266413 21630 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 08:49:12 dist-test-slave-6zbq krb5kdc[20982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092552, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.190@KRBTEST.COM for HTTP/127.4.159.148@KRBTEST.COM
I20260807 08:49:12.322645 21630 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/minicluster-data/master-0/data/instance:
uuid: "f0b0faa844f546198999b93ea74f18f5"
format_stamp: "Formatted at 2026-08-07 08:49:12 on dist-test-slave-6zbq"
server_key: "f59effb3796647691dc2c8f8e449549f"
server_key_iv: "ec59a123c45fc245a1b0409b95418e6a"
server_key_version: "kuduclusterkey@0"
I20260807 08:49:12.323153 21630 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/minicluster-data/master-0/wal/instance:
uuid: "f0b0faa844f546198999b93ea74f18f5"
format_stamp: "Formatted at 2026-08-07 08:49:12 on dist-test-slave-6zbq"
server_key: "f59effb3796647691dc2c8f8e449549f"
server_key_iv: "ec59a123c45fc245a1b0409b95418e6a"
server_key_version: "kuduclusterkey@0"
I20260807 08:49:12.325151 21630 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260807 08:49:12.339071 21657 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:49:12.339301 21630 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 08:49:12.339398 21630 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/minicluster-data/master-0/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/minicluster-data/master-0/wal
uuid: "f0b0faa844f546198999b93ea74f18f5"
format_stamp: "Formatted at 2026-08-07 08:49:12 on dist-test-slave-6zbq"
server_key: "f59effb3796647691dc2c8f8e449549f"
server_key_iv: "ec59a123c45fc245a1b0409b95418e6a"
server_key_version: "kuduclusterkey@0"
I20260807 08:49:12.339494 21630 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:49:12.352907 21630 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:49:12.355098 21630 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:49:12.355268 21630 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:49:12.364763 21630 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.190:41811
I20260807 08:49:12.364791 21738 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.190:41811 every 8 connection(s)
I20260807 08:49:12.365245 21630 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
I20260807 08:49:12.366035 21739 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:49:12.368809 21739 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f0b0faa844f546198999b93ea74f18f5: Bootstrap starting.
I20260807 08:49:12.369477 21739 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f0b0faa844f546198999b93ea74f18f5: Neither blocks nor log segments found. Creating new log.
I20260807 08:49:12.369966 21739 log.cc:824] T 00000000000000000000000000000000 P f0b0faa844f546198999b93ea74f18f5: Log is configured to *not* fsync() on all Append() calls
I20260807 08:49:12.370714 21739 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f0b0faa844f546198999b93ea74f18f5: No bootstrap required, opened a new log
I20260807 08:49:12.372081 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 21630
I20260807 08:49:12.372206 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/minicluster-data/master-0/wal/instance
I20260807 08:49:12.372418 4734 external_mini_cluster.cc:1428] Setting key dfb4d599534c6d4337e8e2d2ce637eb5
I20260807 08:49:12.373553 21739 raft_consensus.cc:348] T 00000000000000000000000000000000 P f0b0faa844f546198999b93ea74f18f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0b0faa844f546198999b93ea74f18f5" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 41811 } }
I20260807 08:49:12.373694 21739 raft_consensus.cc:374] T 00000000000000000000000000000000 P f0b0faa844f546198999b93ea74f18f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:49:12.373728 21739 raft_consensus.cc:729] T 00000000000000000000000000000000 P f0b0faa844f546198999b93ea74f18f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0b0faa844f546198999b93ea74f18f5, State: Initialized, Role: FOLLOWER
I20260807 08:49:12.373878 21739 consensus_queue.cc:260] T 00000000000000000000000000000000 P f0b0faa844f546198999b93ea74f18f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0b0faa844f546198999b93ea74f18f5" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 41811 } }
I20260807 08:49:12.373963 21739 raft_consensus.cc:388] T 00000000000000000000000000000000 P f0b0faa844f546198999b93ea74f18f5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:49:12.373994 21739 raft_consensus.cc:482] T 00000000000000000000000000000000 P f0b0faa844f546198999b93ea74f18f5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:49:12.374040 21739 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f0b0faa844f546198999b93ea74f18f5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:49:12.374868 21739 raft_consensus.cc:504] T 00000000000000000000000000000000 P f0b0faa844f546198999b93ea74f18f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0b0faa844f546198999b93ea74f18f5" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 41811 } }
I20260807 08:49:12.374955 21739 leader_election.cc:302] T 00000000000000000000000000000000 P f0b0faa844f546198999b93ea74f18f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f0b0faa844f546198999b93ea74f18f5; no voters:
I20260807 08:49:12.375097 21739 leader_election.cc:288] T 00000000000000000000000000000000 P f0b0faa844f546198999b93ea74f18f5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:49:12.375205 21745 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f0b0faa844f546198999b93ea74f18f5 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:49:12.375448 21739 sys_catalog.cc:564] T 00000000000000000000000000000000 P f0b0faa844f546198999b93ea74f18f5 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:49:12.375491 21745 raft_consensus.cc:686] T 00000000000000000000000000000000 P f0b0faa844f546198999b93ea74f18f5 [term 1 LEADER]: Becoming Leader. State: Replica: f0b0faa844f546198999b93ea74f18f5, State: Running, Role: LEADER
I20260807 08:49:12.375667 21745 consensus_queue.cc:237] T 00000000000000000000000000000000 P f0b0faa844f546198999b93ea74f18f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0b0faa844f546198999b93ea74f18f5" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 41811 } }
I20260807 08:49:12.376176 21746 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0b0faa844f546198999b93ea74f18f5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f0b0faa844f546198999b93ea74f18f5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0b0faa844f546198999b93ea74f18f5" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 41811 } } }
I20260807 08:49:12.376276 21746 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0b0faa844f546198999b93ea74f18f5 [sys.catalog]: This master's current role is: LEADER
I20260807 08:49:12.376190 21747 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0b0faa844f546198999b93ea74f18f5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f0b0faa844f546198999b93ea74f18f5. Latest consensus state: current_term: 1 leader_uuid: "f0b0faa844f546198999b93ea74f18f5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0b0faa844f546198999b93ea74f18f5" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 41811 } } }
I20260807 08:49:12.376617 21747 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0b0faa844f546198999b93ea74f18f5 [sys.catalog]: This master's current role is: LEADER
I20260807 08:49:12.376704 21751 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:49:12.377395 21751 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 07 08:49:12 dist-test-slave-6zbq krb5kdc[20982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092552, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.159.190@KRBTEST.COM
I20260807 08:49:12.383564 21739 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260807 08:49:12.383896 21744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:49:12.372948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41700 (local address 127.4.159.190:41811)
0807 08:49:12.373248 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0807 08:49:12.373257 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 08:49:12.373311 (+ 54us) server_negotiation.cc:407] Connection header received
0807 08:49:12.373552 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:49:12.373561 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:49:12.373615 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:49:12.373728 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:49:12.374561 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:49:12.376070 (+ 1509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:49:12.377142 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:49:12.377794 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:49:12.377896 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:49:12.380087 (+ 2191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:49:12.380091 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:49:12.380091 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:49:12.380103 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:49:12.381930 (+ 1827us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:49:12.382241 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:49:12.382244 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:49:12.382245 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:49:12.382420 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:49:12.382607 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:49:12.382608 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:49:12.382609 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:49:12.382935 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:49:12.383141 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0807 08:49:12.383307 (+ 166us) server_negotiation.cc:299] Negotiation successful
0807 08:49:12.383434 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":234,"thread_start_us":120,"threads_started":1}
I20260807 08:49:12.390332 21751 catalog_manager.cc:1347] Generated new cluster ID: 66d15e3874004b23912c565384d00892
I20260807 08:49:12.390414 21751 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:49:12.397958 21751 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:49:12.398677 21751 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:49:12.406365 21751 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f0b0faa844f546198999b93ea74f18f5: Generated new TSK 0
I20260807 08:49:12.406531 21751 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.159.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.159.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.159.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.129.keytab.
Entry for principal kudu/127.4.159.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.129.keytab.
I20260807 08:49:13.138621 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.159.148:40875/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.159.129:0
--local_ip_for_outbound_sockets=127.4.159.129
--webserver_interface=127.4.159.129
--webserver_port=0
--tserver_master_addrs=127.4.159.190:41811
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.129.keytab
--principal=kudu/127.4.159.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:43823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:49:13.266192 21805 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:49:13.266415 21805 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:49:13.266451 21805 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:49:13.267668 21805 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:49:13.267706 21805 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:49:13.267740 21805 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.159.129
I20260807 08:49:13.269251 21805 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:43823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.4.159.148:40875/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.129.keytab
--principal=kudu/127.4.159.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.159.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.159.129
--webserver_port=0
--tserver_master_addrs=127.4.159.190:41811
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21805
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.159.129
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:49:13.269466 21805 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:49:13.269666 21805 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:49:13.276209 21813 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:49:13.276458 21818 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:49:13.276786 21812 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:49:13.276844 21805 server_base.cc:1034] running on GCE node
I20260807 08:49:13.277086 21805 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:49:13.277355 21805 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:49:13.278560 21805 hybrid_clock.cc:648] HybridClock initialized: now 1786092553278525 us; error 48 us; skew 500 ppm
Aug 07 08:49:13 dist-test-slave-6zbq krb5kdc[20982](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092553, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:49:13.286700 21805 init.cc:377] Logged in from keytab as kudu/127.4.159.129@KRBTEST.COM (short username kudu)
I20260807 08:49:13.287101 21805 webserver.cc:460] Webserver started at http://127.4.159.129:45325/ using document root <none> and password file <none>
I20260807 08:49:13.287292 21805 fs_manager.cc:359] Metadata directory not provided
I20260807 08:49:13.287338 21805 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:49:13.287457 21805 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 08:49:13 dist-test-slave-6zbq krb5kdc[20982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092553, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.129@KRBTEST.COM for HTTP/127.4.159.148@KRBTEST.COM
I20260807 08:49:13.301422 21805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/minicluster-data/ts-0/data/instance:
uuid: "58b05d20c13e4055955cb7591d50f7b1"
format_stamp: "Formatted at 2026-08-07 08:49:13 on dist-test-slave-6zbq"
server_key: "f57d25f91cf7ac5ef3ba3a637f7c1b03"
server_key_iv: "48858d68ac3750855a9ebd102a912616"
server_key_version: "kuduclusterkey@0"
I20260807 08:49:13.302098 21805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/minicluster-data/ts-0/wal/instance:
uuid: "58b05d20c13e4055955cb7591d50f7b1"
format_stamp: "Formatted at 2026-08-07 08:49:13 on dist-test-slave-6zbq"
server_key: "f57d25f91cf7ac5ef3ba3a637f7c1b03"
server_key_iv: "48858d68ac3750855a9ebd102a912616"
server_key_version: "kuduclusterkey@0"
I20260807 08:49:13.306123 21805 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.001s
I20260807 08:49:13.324076 21832 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:49:13.324261 21805 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.000s sys 0.000s
I20260807 08:49:13.324362 21805 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/minicluster-data/ts-0/wal
uuid: "58b05d20c13e4055955cb7591d50f7b1"
format_stamp: "Formatted at 2026-08-07 08:49:13 on dist-test-slave-6zbq"
server_key: "f57d25f91cf7ac5ef3ba3a637f7c1b03"
server_key_iv: "48858d68ac3750855a9ebd102a912616"
server_key_version: "kuduclusterkey@0"
I20260807 08:49:13.324460 21805 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:49:13.336001 21805 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:49:13.338308 21805 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:49:13.338536 21805 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:49:13.338909 21805 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:49:13.339363 21805 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:49:13.339447 21805 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:49:13.339540 21805 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:49:13.339614 21805 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:49:13.353513 21805 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.129:34513
I20260807 08:49:13.354005 21805 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/minicluster-data/ts-0/data/info.pb
I20260807 08:49:13.354498 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 21805
I20260807 08:49:13.354609 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/minicluster-data/ts-0/wal/instance
I20260807 08:49:13.354825 4734 external_mini_cluster.cc:1428] Setting key df570fd336dd8674d990104955563129
I20260807 08:49:13.356540 22015 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.129:34513 every 8 connection(s)
Aug 07 08:49:13 dist-test-slave-6zbq krb5kdc[20982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092553, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.129@KRBTEST.COM for kudu/127.4.159.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.4.159.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.159.130@KRBTEST.COM" created.
I20260807 08:49:13.371752 22021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:49:13.355712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:41811 (local address 127.4.159.129:49075)
0807 08:49:13.356523 (+ 811us) negotiation.cc:107] Waiting for socket to connect
0807 08:49:13.356566 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 08:49:13.357573 (+ 1007us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:49:13.359011 (+ 1438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:49:13.359016 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:49:13.359270 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:49:13.359582 (+ 312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:49:13.359589 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:49:13.360624 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:49:13.360625 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:49:13.362041 (+ 1416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:49:13.362044 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:49:13.362618 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:49:13.362620 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:49:13.362757 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:49:13.363323 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:49:13.363333 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:49:13.367615 (+ 4282us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:49:13.369914 (+ 2299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:49:13.369921 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:49:13.369923 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:49:13.370215 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:49:13.370516 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:49:13.370517 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:49:13.370517 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:49:13.370659 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:49:13.371020 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:49:13.371027 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:49:13.371183 (+ 156us) client_negotiation.cc:769] Sending connection context
0807 08:49:13.371310 (+ 127us) client_negotiation.cc:240] Negotiation successful
0807 08:49:13.371471 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":749,"thread_start_us":603,"threads_started":1}
I20260807 08:49:13.372280 22020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:49:13.355706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.129:49075 (local address 127.4.159.190:41811)
0807 08:49:13.358735 (+ 3029us) server_negotiation.cc:206] Beginning negotiation
0807 08:49:13.358737 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:49:13.358748 (+ 11us) server_negotiation.cc:407] Connection header received
0807 08:49:13.358774 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:49:13.358775 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:49:13.358789 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:49:13.358866 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:49:13.359699 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:49:13.360505 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:49:13.362141 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:49:13.362471 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:49:13.362538 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:49:13.367728 (+ 5190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:49:13.367729 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:49:13.367730 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:49:13.367738 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:49:13.369817 (+ 2079us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:49:13.370363 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:49:13.370364 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:49:13.370365 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:49:13.370433 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:49:13.370738 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:49:13.370739 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:49:13.370739 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:49:13.370918 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:49:13.370980 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 08:49:13.372105 (+ 1125us) server_negotiation.cc:299] Negotiation successful
0807 08:49:13.372191 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2967,"thread_start_us":74,"threads_started":1}
I20260807 08:49:13.373085 22017 heartbeater.cc:347] Connected to a master server at 127.4.159.190:41811
I20260807 08:49:13.373180 22017 heartbeater.cc:464] Registering TS with master...
I20260807 08:49:13.373379 22017 heartbeater.cc:511] Master 127.4.159.190:41811 requested a full tablet report, sending...
I20260807 08:49:13.373991 21682 ts_manager.cc:194] Registered new tserver with Master: 58b05d20c13e4055955cb7591d50f7b1 (127.4.159.129:34513)
I20260807 08:49:13.374854 21682 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.159.129@KRBTEST.COM'} at 127.4.159.129:49075
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.159.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.130.keytab.
Entry for principal kudu/127.4.159.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.130.keytab.
I20260807 08:49:13.395495 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.159.148:40875/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.159.130:0
--local_ip_for_outbound_sockets=127.4.159.130
--webserver_interface=127.4.159.130
--webserver_port=0
--tserver_master_addrs=127.4.159.190:41811
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.130.keytab
--principal=kudu/127.4.159.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:43823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:49:13.522553 22027 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:49:13.522746 22027 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:49:13.522778 22027 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:49:13.523936 22027 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:49:13.523978 22027 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:49:13.524013 22027 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.159.130
I20260807 08:49:13.525539 22027 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:43823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.4.159.148:40875/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.130.keytab
--principal=kudu/127.4.159.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.159.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.159.130
--webserver_port=0
--tserver_master_addrs=127.4.159.190:41811
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22027
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.159.130
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:49:13.525763 22027 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:49:13.525995 22027 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:49:13.531621 22037 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:49:13.531600 22036 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:49:13.531877 22039 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:49:13.532733 22027 server_base.cc:1034] running on GCE node
I20260807 08:49:13.532918 22027 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:49:13.533134 22027 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:49:13.534272 22027 hybrid_clock.cc:648] HybridClock initialized: now 1786092553534257 us; error 26 us; skew 500 ppm
Aug 07 08:49:13 dist-test-slave-6zbq krb5kdc[20982](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092553, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:49:13.537212 22027 init.cc:377] Logged in from keytab as kudu/127.4.159.130@KRBTEST.COM (short username kudu)
I20260807 08:49:13.537624 22027 webserver.cc:460] Webserver started at http://127.4.159.130:36119/ using document root <none> and password file <none>
I20260807 08:49:13.537822 22027 fs_manager.cc:359] Metadata directory not provided
I20260807 08:49:13.537868 22027 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:49:13.537993 22027 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 08:49:13 dist-test-slave-6zbq krb5kdc[20982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092553, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.130@KRBTEST.COM for HTTP/127.4.159.148@KRBTEST.COM
I20260807 08:49:13.558662 22027 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/minicluster-data/ts-1/data/instance:
uuid: "f25bc24250484e5c9481ff5b2666f101"
format_stamp: "Formatted at 2026-08-07 08:49:13 on dist-test-slave-6zbq"
server_key: "2a9978cf6915fcb7c318db337f7d3d8a"
server_key_iv: "3788b79fed9fad87f018358c4a04a6a3"
server_key_version: "kuduclusterkey@0"
I20260807 08:49:13.559213 22027 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/minicluster-data/ts-1/wal/instance:
uuid: "f25bc24250484e5c9481ff5b2666f101"
format_stamp: "Formatted at 2026-08-07 08:49:13 on dist-test-slave-6zbq"
server_key: "2a9978cf6915fcb7c318db337f7d3d8a"
server_key_iv: "3788b79fed9fad87f018358c4a04a6a3"
server_key_version: "kuduclusterkey@0"
I20260807 08:49:13.566846 22027 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.001s sys 0.000s
Aug 07 08:49:13 dist-test-slave-6zbq krb5kdc[20982](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786092553, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:49:13.580065 22054 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:49:13.580281 22027 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:49:13.580371 22027 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/minicluster-data/ts-1/wal
uuid: "f25bc24250484e5c9481ff5b2666f101"
format_stamp: "Formatted at 2026-08-07 08:49:13 on dist-test-slave-6zbq"
server_key: "2a9978cf6915fcb7c318db337f7d3d8a"
server_key_iv: "3788b79fed9fad87f018358c4a04a6a3"
server_key_version: "kuduclusterkey@0"
I20260807 08:49:13.580443 22027 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:49:13.598443 22027 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:49:13.600839 22027 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:49:13.601019 22027 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:49:13.601385 22027 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:49:13.601869 22027 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:49:13.601961 22027 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:49:13.602037 22027 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:49:13.602100 22027 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:49:13.616990 22027 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.130:33957
I20260807 08:49:13.617496 22027 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/minicluster-data/ts-1/data/info.pb
I20260807 08:49:13.617780 22229 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.130:33957 every 8 connection(s)
Aug 07 08:49:13 dist-test-slave-6zbq krb5kdc[20982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092553, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.130@KRBTEST.COM for kudu/127.4.159.190@KRBTEST.COM
I20260807 08:49:13.626520 22020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:49:13.618838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.130:58839 (local address 127.4.159.190:41811)
0807 08:49:13.618887 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0807 08:49:13.618889 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:49:13.619043 (+ 154us) server_negotiation.cc:407] Connection header received
0807 08:49:13.619723 (+ 680us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:49:13.619725 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:49:13.619743 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:49:13.619793 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:49:13.620391 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:49:13.620877 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:49:13.621856 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:49:13.622047 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:49:13.622102 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:49:13.624007 (+ 1905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:49:13.624009 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:49:13.624009 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:49:13.624017 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:49:13.625289 (+ 1272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:49:13.625638 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:49:13.625639 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:49:13.625640 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:49:13.625692 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:49:13.625905 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:49:13.625906 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:49:13.625906 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:49:13.626011 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:49:13.626066 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 08:49:13.626321 (+ 255us) server_negotiation.cc:299] Negotiation successful
0807 08:49:13.626431 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 08:49:13.626653 22233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:49:13.618561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:41811 (local address 127.4.159.130:58839)
0807 08:49:13.618915 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0807 08:49:13.618958 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 08:49:13.619617 (+ 659us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:49:13.619872 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:49:13.619876 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:49:13.620103 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:49:13.620296 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:49:13.620301 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:49:13.620958 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:49:13.620960 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:49:13.621770 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:49:13.621773 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:49:13.622120 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:49:13.622121 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:49:13.622237 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:49:13.622705 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:49:13.622714 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:49:13.623899 (+ 1185us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:49:13.625365 (+ 1466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:49:13.625371 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:49:13.625373 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:49:13.625571 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:49:13.625745 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:49:13.625746 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:49:13.625746 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:49:13.625849 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:49:13.626067 (+ 218us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:49:13.626073 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:49:13.626206 (+ 133us) client_negotiation.cc:769] Sending connection context
0807 08:49:13.626291 (+ 85us) client_negotiation.cc:240] Negotiation successful
0807 08:49:13.626446 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":89,"threads_started":1}
I20260807 08:49:13.626977 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 22027
I20260807 08:49:13.627081 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/minicluster-data/ts-1/wal/instance
I20260807 08:49:13.627259 4734 external_mini_cluster.cc:1428] Setting key 00b352e5433fd69de932f119555717a0
I20260807 08:49:13.627313 22230 heartbeater.cc:347] Connected to a master server at 127.4.159.190:41811
I20260807 08:49:13.627388 22230 heartbeater.cc:464] Registering TS with master...
I20260807 08:49:13.627556 22230 heartbeater.cc:511] Master 127.4.159.190:41811 requested a full tablet report, sending...
I20260807 08:49:13.627935 21681 ts_manager.cc:194] Registered new tserver with Master: f25bc24250484e5c9481ff5b2666f101 (127.4.159.130:33957)
I20260807 08:49:13.628417 21681 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.159.130@KRBTEST.COM'} at 127.4.159.130:58839
WARNING: no policy specified for kudu/127.4.159.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.159.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.159.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.131.keytab.
Entry for principal kudu/127.4.159.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.131.keytab.
I20260807 08:49:13.652347 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.159.148:40875/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.159.131:0
--local_ip_for_outbound_sockets=127.4.159.131
--webserver_interface=127.4.159.131
--webserver_port=0
--tserver_master_addrs=127.4.159.190:41811
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.131.keytab
--principal=kudu/127.4.159.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:43823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:49:13.799422 22238 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:49:13.799678 22238 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:49:13.799714 22238 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:49:13.801436 22238 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:49:13.801497 22238 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:49:13.801537 22238 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.159.131
I20260807 08:49:13.803944 22238 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:43823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.4.159.148:40875/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.131.keytab
--principal=kudu/127.4.159.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.159.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.159.131
--webserver_port=0
--tserver_master_addrs=127.4.159.190:41811
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22238
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.159.131
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:49:13.804167 22238 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:49:13.804473 22238 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:49:13.811156 22255 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:49:13.811232 22259 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:49:13.811514 22254 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:49:13.812362 22238 server_base.cc:1034] running on GCE node
I20260807 08:49:13.812572 22238 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:49:13.812839 22238 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:49:13.814023 22238 hybrid_clock.cc:648] HybridClock initialized: now 1786092553814006 us; error 24 us; skew 500 ppm
Aug 07 08:49:13 dist-test-slave-6zbq krb5kdc[20982](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092553, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:49:13.817678 22238 init.cc:377] Logged in from keytab as kudu/127.4.159.131@KRBTEST.COM (short username kudu)
I20260807 08:49:13.818077 22238 webserver.cc:460] Webserver started at http://127.4.159.131:42273/ using document root <none> and password file <none>
I20260807 08:49:13.818300 22238 fs_manager.cc:359] Metadata directory not provided
I20260807 08:49:13.818373 22238 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:49:13.818502 22238 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 08:49:13 dist-test-slave-6zbq krb5kdc[20982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092553, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.131@KRBTEST.COM for HTTP/127.4.159.148@KRBTEST.COM
I20260807 08:49:13.831699 22238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/minicluster-data/ts-2/data/instance:
uuid: "c1e154de2edc46569817f95faa42dd4c"
format_stamp: "Formatted at 2026-08-07 08:49:13 on dist-test-slave-6zbq"
server_key: "44ea8ca4261527b132afa6a5e841d7a3"
server_key_iv: "bed726cf72d9a365d5647fcd7a1975cc"
server_key_version: "kuduclusterkey@0"
I20260807 08:49:13.832135 22238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/minicluster-data/ts-2/wal/instance:
uuid: "c1e154de2edc46569817f95faa42dd4c"
format_stamp: "Formatted at 2026-08-07 08:49:13 on dist-test-slave-6zbq"
server_key: "44ea8ca4261527b132afa6a5e841d7a3"
server_key_iv: "bed726cf72d9a365d5647fcd7a1975cc"
server_key_version: "kuduclusterkey@0"
I20260807 08:49:13.833992 22238 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260807 08:49:13.846460 22276 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:49:13.846640 22238 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:49:13.846712 22238 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/minicluster-data/ts-2/wal
uuid: "c1e154de2edc46569817f95faa42dd4c"
format_stamp: "Formatted at 2026-08-07 08:49:13 on dist-test-slave-6zbq"
server_key: "44ea8ca4261527b132afa6a5e841d7a3"
server_key_iv: "bed726cf72d9a365d5647fcd7a1975cc"
server_key_version: "kuduclusterkey@0"
I20260807 08:49:13.846778 22238 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:49:13.875119 22238 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:49:13.877110 22238 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:49:13.877252 22238 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:49:13.877482 22238 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:49:13.877869 22238 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:49:13.877960 22238 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:49:13.878033 22238 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:49:13.878091 22238 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:49:13.891811 22238 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.131:34445
I20260807 08:49:13.891847 22456 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.131:34445 every 8 connection(s)
I20260807 08:49:13.892235 22238 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/minicluster-data/ts-2/data/info.pb
I20260807 08:49:13.899210 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 22238
I20260807 08:49:13.899334 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/minicluster-data/ts-2/wal/instance
Aug 07 08:49:13 dist-test-slave-6zbq krb5kdc[20982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092553, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.131@KRBTEST.COM for kudu/127.4.159.190@KRBTEST.COM
I20260807 08:49:13.899523 4734 external_mini_cluster.cc:1428] Setting key 6ec0a68e0c3f0d9b18858c8fc26bfd89
I20260807 08:49:13.902489 22020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:49:13.893019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.131:45111 (local address 127.4.159.190:41811)
0807 08:49:13.893082 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0807 08:49:13.893084 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:49:13.893273 (+ 189us) server_negotiation.cc:407] Connection header received
0807 08:49:13.894209 (+ 936us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:49:13.894210 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:49:13.894219 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:49:13.894266 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:49:13.895209 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:49:13.895687 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:49:13.897088 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:49:13.897265 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:49:13.897308 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:49:13.899907 (+ 2599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:49:13.899909 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:49:13.899909 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:49:13.899918 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:49:13.901253 (+ 1335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:49:13.901601 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:49:13.901602 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:49:13.901602 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:49:13.901661 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:49:13.901872 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:49:13.901873 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:49:13.901873 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:49:13.901985 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:49:13.902041 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 08:49:13.902292 (+ 251us) server_negotiation.cc:299] Negotiation successful
0807 08:49:13.902401 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260807 08:49:13.902626 22463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:49:13.892989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:41811 (local address 127.4.159.131:45111)
0807 08:49:13.893140 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0807 08:49:13.893187 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 08:49:13.894098 (+ 911us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:49:13.894378 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:49:13.894382 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:49:13.894756 (+ 374us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:49:13.895126 (+ 370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:49:13.895134 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:49:13.895775 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:49:13.895776 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:49:13.897001 (+ 1225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:49:13.897005 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:49:13.897337 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:49:13.897338 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:49:13.897532 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:49:13.898077 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:49:13.898087 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:49:13.899821 (+ 1734us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:49:13.901326 (+ 1505us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:49:13.901333 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:49:13.901335 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:49:13.901535 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:49:13.901712 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:49:13.901713 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:49:13.901713 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:49:13.901818 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:49:13.902042 (+ 224us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:49:13.902047 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:49:13.902174 (+ 127us) client_negotiation.cc:769] Sending connection context
0807 08:49:13.902261 (+ 87us) client_negotiation.cc:240] Negotiation successful
0807 08:49:13.902418 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":74,"threads_started":1}
I20260807 08:49:13.903064 22457 heartbeater.cc:347] Connected to a master server at 127.4.159.190:41811
I20260807 08:49:13.903143 22457 heartbeater.cc:464] Registering TS with master...
I20260807 08:49:13.903311 22457 heartbeater.cc:511] Master 127.4.159.190:41811 requested a full tablet report, sending...
I20260807 08:49:13.903651 21681 ts_manager.cc:194] Registered new tserver with Master: c1e154de2edc46569817f95faa42dd4c (127.4.159.131:34445)
I20260807 08:49:13.903995 21681 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.159.131@KRBTEST.COM'} at 127.4.159.131:45111
I20260807 08:49:13.911113 4734 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 08:49:13.919260 22020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:49:13.912744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41714 (local address 127.4.159.190:41811)
0807 08:49:13.912787 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0807 08:49:13.912788 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 08:49:13.912931 (+ 143us) server_negotiation.cc:407] Connection header received
0807 08:49:13.913001 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:49:13.913002 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:49:13.913010 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:49:13.913049 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:49:13.913758 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:49:13.914194 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:49:13.915236 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:49:13.915443 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:49:13.915491 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:49:13.916861 (+ 1370us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:49:13.916862 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:49:13.916863 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:49:13.916869 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:49:13.918142 (+ 1273us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:49:13.918595 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:49:13.918596 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:49:13.918596 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:49:13.918642 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:49:13.918854 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:49:13.918855 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:49:13.918855 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:49:13.918940 (+ 85us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:49:13.919003 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0807 08:49:13.919149 (+ 146us) server_negotiation.cc:299] Negotiation successful
0807 08:49:13.919201 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 08:49:13.921052 21681 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41714:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 08:49:13.923561 21681 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:49:13.931691 22483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:49:13.926127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.129:34513 (local address 127.0.0.1:49978)
0807 08:49:13.926762 (+ 635us) negotiation.cc:107] Waiting for socket to connect
0807 08:49:13.926909 (+ 147us) client_negotiation.cc:174] Beginning negotiation
0807 08:49:13.926964 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:49:13.927245 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:49:13.927247 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:49:13.927249 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:49:13.927292 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:49:13.927293 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:49:13.928545 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:49:13.928546 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:49:13.930048 (+ 1502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:49:13.930051 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:49:13.931411 (+ 1360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:49:13.931412 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:49:13.931485 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:49:13.931498 (+ 13us) client_negotiation.cc:769] Sending connection context
0807 08:49:13.931564 (+ 66us) client_negotiation.cc:240] Negotiation successful
0807 08:49:13.931615 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":592,"thread_start_us":491,"threads_started":1}
I20260807 08:49:13.932037 22478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:49:13.926331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49978 (local address 127.4.159.129:34513)
0807 08:49:13.926486 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0807 08:49:13.926491 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:49:13.926983 (+ 492us) server_negotiation.cc:407] Connection header received
0807 08:49:13.927107 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:49:13.927110 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:49:13.927127 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:49:13.927173 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:49:13.927356 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:49:13.928454 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:49:13.930146 (+ 1692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:49:13.931260 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:49:13.931314 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:49:13.931375 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 08:49:13.931929 (+ 554us) server_negotiation.cc:299] Negotiation successful
0807 08:49:13.931973 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":86,"threads_started":1}
I20260807 08:49:13.932047 22482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:49:13.926505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58714 (local address 127.4.159.131:34445)
0807 08:49:13.927585 (+ 1080us) server_negotiation.cc:206] Beginning negotiation
0807 08:49:13.927589 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:49:13.927598 (+ 9us) server_negotiation.cc:407] Connection header received
0807 08:49:13.927640 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:49:13.927642 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:49:13.927657 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:49:13.927900 (+ 243us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:49:13.928058 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:49:13.929215 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:49:13.930609 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:49:13.931389 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:49:13.931444 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:49:13.931587 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0807 08:49:13.931882 (+ 295us) server_negotiation.cc:299] Negotiation successful
0807 08:49:13.931921 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1049,"thread_start_us":127,"threads_started":1}
I20260807 08:49:13.932247 22479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:49:13.926330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59840 (local address 127.4.159.130:33957)
0807 08:49:13.926852 (+ 522us) server_negotiation.cc:206] Beginning negotiation
0807 08:49:13.926856 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:49:13.927499 (+ 643us) server_negotiation.cc:407] Connection header received
0807 08:49:13.927582 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:49:13.927584 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:49:13.927602 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:49:13.927686 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:49:13.927988 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:49:13.929152 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:49:13.930580 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:49:13.931490 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:49:13.931565 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:49:13.931718 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0807 08:49:13.932078 (+ 360us) server_negotiation.cc:299] Negotiation successful
0807 08:49:13.932123 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":465,"thread_start_us":144,"threads_started":1}
I20260807 08:49:13.931826 22485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:49:13.926131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.131:34445 (local address 127.0.0.1:58714)
0807 08:49:13.926830 (+ 699us) negotiation.cc:107] Waiting for socket to connect
0807 08:49:13.926859 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0807 08:49:13.926929 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:49:13.927854 (+ 925us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:49:13.927855 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:49:13.927857 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:49:13.927984 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:49:13.927986 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:49:13.929301 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:49:13.929302 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:49:13.930479 (+ 1177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:49:13.930482 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:49:13.931629 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:49:13.931630 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:49:13.931687 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:49:13.931690 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 08:49:13.931736 (+ 46us) client_negotiation.cc:240] Negotiation successful
0807 08:49:13.931769 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":652,"mutex_wait_us":102,"thread_start_us":592,"threads_started":1}
I20260807 08:49:13.932010 22481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:49:13.926142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.130:33957 (local address 127.0.0.1:59840)
0807 08:49:13.927414 (+ 1272us) negotiation.cc:107] Waiting for socket to connect
0807 08:49:13.927425 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 08:49:13.927480 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:49:13.927804 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:49:13.927805 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:49:13.927807 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:49:13.927894 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:49:13.927897 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:49:13.929255 (+ 1358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:49:13.929256 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:49:13.930457 (+ 1201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:49:13.930460 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:49:13.931771 (+ 1311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:49:13.931773 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:49:13.931837 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:49:13.931842 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 08:49:13.931897 (+ 55us) client_negotiation.cc:240] Negotiation successful
0807 08:49:13.931936 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1254,"mutex_wait_us":38,"thread_start_us":508,"threads_started":1}
I20260807 08:49:13.933009 21908 tablet_service.cc:1467] Processing CreateTablet for tablet 2d91069a9a5f4eccad55af83cacfd1a6 (DEFAULT_TABLE table=test-table [id=14832a99fd2242248611bc1e9f4aed8e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:49:13.932983 22360 tablet_service.cc:1467] Processing CreateTablet for tablet 2d91069a9a5f4eccad55af83cacfd1a6 (DEFAULT_TABLE table=test-table [id=14832a99fd2242248611bc1e9f4aed8e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:49:13.933336 21908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d91069a9a5f4eccad55af83cacfd1a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:49:13.933357 22360 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d91069a9a5f4eccad55af83cacfd1a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:49:13.937556 22487 tablet_bootstrap.cc:492] T 2d91069a9a5f4eccad55af83cacfd1a6 P 58b05d20c13e4055955cb7591d50f7b1: Bootstrap starting.
I20260807 08:49:13.937767 22127 tablet_service.cc:1467] Processing CreateTablet for tablet 2d91069a9a5f4eccad55af83cacfd1a6 (DEFAULT_TABLE table=test-table [id=14832a99fd2242248611bc1e9f4aed8e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:49:13.938162 22127 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d91069a9a5f4eccad55af83cacfd1a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:49:13.938566 22487 tablet_bootstrap.cc:654] T 2d91069a9a5f4eccad55af83cacfd1a6 P 58b05d20c13e4055955cb7591d50f7b1: Neither blocks nor log segments found. Creating new log.
I20260807 08:49:13.939206 22487 log.cc:824] T 2d91069a9a5f4eccad55af83cacfd1a6 P 58b05d20c13e4055955cb7591d50f7b1: Log is configured to *not* fsync() on all Append() calls
I20260807 08:49:13.939944 22487 tablet_bootstrap.cc:492] T 2d91069a9a5f4eccad55af83cacfd1a6 P 58b05d20c13e4055955cb7591d50f7b1: No bootstrap required, opened a new log
I20260807 08:49:13.939996 22487 ts_tablet_manager.cc:1397] T 2d91069a9a5f4eccad55af83cacfd1a6 P 58b05d20c13e4055955cb7591d50f7b1: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 08:49:13.943578 22487 raft_consensus.cc:348] T 2d91069a9a5f4eccad55af83cacfd1a6 P 58b05d20c13e4055955cb7591d50f7b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1e154de2edc46569817f95faa42dd4c" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 34445 } } peers { permanent_uuid: "58b05d20c13e4055955cb7591d50f7b1" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 34513 } } peers { permanent_uuid: "f25bc24250484e5c9481ff5b2666f101" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 33957 } }
I20260807 08:49:13.943737 22487 raft_consensus.cc:374] T 2d91069a9a5f4eccad55af83cacfd1a6 P 58b05d20c13e4055955cb7591d50f7b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:49:13.943763 22487 raft_consensus.cc:729] T 2d91069a9a5f4eccad55af83cacfd1a6 P 58b05d20c13e4055955cb7591d50f7b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58b05d20c13e4055955cb7591d50f7b1, State: Initialized, Role: FOLLOWER
I20260807 08:49:13.943879 22487 consensus_queue.cc:260] T 2d91069a9a5f4eccad55af83cacfd1a6 P 58b05d20c13e4055955cb7591d50f7b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1e154de2edc46569817f95faa42dd4c" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 34445 } } peers { permanent_uuid: "58b05d20c13e4055955cb7591d50f7b1" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 34513 } } peers { permanent_uuid: "f25bc24250484e5c9481ff5b2666f101" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 33957 } }
I20260807 08:49:13.944162 22487 ts_tablet_manager.cc:1428] T 2d91069a9a5f4eccad55af83cacfd1a6 P 58b05d20c13e4055955cb7591d50f7b1: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260807 08:49:13.944324 22017 heartbeater.cc:503] Master 127.4.159.190:41811 was elected leader, sending a full tablet report...
I20260807 08:49:13.946574 22490 tablet_bootstrap.cc:492] T 2d91069a9a5f4eccad55af83cacfd1a6 P c1e154de2edc46569817f95faa42dd4c: Bootstrap starting.
I20260807 08:49:13.948128 22490 tablet_bootstrap.cc:654] T 2d91069a9a5f4eccad55af83cacfd1a6 P c1e154de2edc46569817f95faa42dd4c: Neither blocks nor log segments found. Creating new log.
I20260807 08:49:13.948689 22490 log.cc:824] T 2d91069a9a5f4eccad55af83cacfd1a6 P c1e154de2edc46569817f95faa42dd4c: Log is configured to *not* fsync() on all Append() calls
I20260807 08:49:13.950765 22494 tablet_bootstrap.cc:492] T 2d91069a9a5f4eccad55af83cacfd1a6 P f25bc24250484e5c9481ff5b2666f101: Bootstrap starting.
I20260807 08:49:13.952145 22494 tablet_bootstrap.cc:654] T 2d91069a9a5f4eccad55af83cacfd1a6 P f25bc24250484e5c9481ff5b2666f101: Neither blocks nor log segments found. Creating new log.
I20260807 08:49:13.952608 22494 log.cc:824] T 2d91069a9a5f4eccad55af83cacfd1a6 P f25bc24250484e5c9481ff5b2666f101: Log is configured to *not* fsync() on all Append() calls
I20260807 08:49:13.953334 22494 tablet_bootstrap.cc:492] T 2d91069a9a5f4eccad55af83cacfd1a6 P f25bc24250484e5c9481ff5b2666f101: No bootstrap required, opened a new log
I20260807 08:49:13.953343 22490 tablet_bootstrap.cc:492] T 2d91069a9a5f4eccad55af83cacfd1a6 P c1e154de2edc46569817f95faa42dd4c: No bootstrap required, opened a new log
I20260807 08:49:13.953399 22490 ts_tablet_manager.cc:1397] T 2d91069a9a5f4eccad55af83cacfd1a6 P c1e154de2edc46569817f95faa42dd4c: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260807 08:49:13.953761 22494 ts_tablet_manager.cc:1397] T 2d91069a9a5f4eccad55af83cacfd1a6 P f25bc24250484e5c9481ff5b2666f101: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 08:49:13.956523 22490 raft_consensus.cc:348] T 2d91069a9a5f4eccad55af83cacfd1a6 P c1e154de2edc46569817f95faa42dd4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1e154de2edc46569817f95faa42dd4c" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 34445 } } peers { permanent_uuid: "58b05d20c13e4055955cb7591d50f7b1" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 34513 } } peers { permanent_uuid: "f25bc24250484e5c9481ff5b2666f101" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 33957 } }
I20260807 08:49:13.956735 22490 raft_consensus.cc:374] T 2d91069a9a5f4eccad55af83cacfd1a6 P c1e154de2edc46569817f95faa42dd4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:49:13.956809 22490 raft_consensus.cc:729] T 2d91069a9a5f4eccad55af83cacfd1a6 P c1e154de2edc46569817f95faa42dd4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1e154de2edc46569817f95faa42dd4c, State: Initialized, Role: FOLLOWER
I20260807 08:49:13.956949 22490 consensus_queue.cc:260] T 2d91069a9a5f4eccad55af83cacfd1a6 P c1e154de2edc46569817f95faa42dd4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1e154de2edc46569817f95faa42dd4c" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 34445 } } peers { permanent_uuid: "58b05d20c13e4055955cb7591d50f7b1" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 34513 } } peers { permanent_uuid: "f25bc24250484e5c9481ff5b2666f101" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 33957 } }
I20260807 08:49:13.956887 22494 raft_consensus.cc:348] T 2d91069a9a5f4eccad55af83cacfd1a6 P f25bc24250484e5c9481ff5b2666f101 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1e154de2edc46569817f95faa42dd4c" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 34445 } } peers { permanent_uuid: "58b05d20c13e4055955cb7591d50f7b1" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 34513 } } peers { permanent_uuid: "f25bc24250484e5c9481ff5b2666f101" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 33957 } }
I20260807 08:49:13.957077 22494 raft_consensus.cc:374] T 2d91069a9a5f4eccad55af83cacfd1a6 P f25bc24250484e5c9481ff5b2666f101 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:49:13.957178 22494 raft_consensus.cc:729] T 2d91069a9a5f4eccad55af83cacfd1a6 P f25bc24250484e5c9481ff5b2666f101 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f25bc24250484e5c9481ff5b2666f101, State: Initialized, Role: FOLLOWER
I20260807 08:49:13.957258 22490 ts_tablet_manager.cc:1428] T 2d91069a9a5f4eccad55af83cacfd1a6 P c1e154de2edc46569817f95faa42dd4c: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20260807 08:49:13.957384 22457 heartbeater.cc:503] Master 127.4.159.190:41811 was elected leader, sending a full tablet report...
I20260807 08:49:13.957590 22494 consensus_queue.cc:260] T 2d91069a9a5f4eccad55af83cacfd1a6 P f25bc24250484e5c9481ff5b2666f101 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1e154de2edc46569817f95faa42dd4c" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 34445 } } peers { permanent_uuid: "58b05d20c13e4055955cb7591d50f7b1" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 34513 } } peers { permanent_uuid: "f25bc24250484e5c9481ff5b2666f101" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 33957 } }
I20260807 08:49:13.958082 22494 ts_tablet_manager.cc:1428] T 2d91069a9a5f4eccad55af83cacfd1a6 P f25bc24250484e5c9481ff5b2666f101: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260807 08:49:13.958110 22230 heartbeater.cc:503] Master 127.4.159.190:41811 was elected leader, sending a full tablet report...
I20260807 08:49:13.973052 22497 raft_consensus.cc:482] T 2d91069a9a5f4eccad55af83cacfd1a6 P c1e154de2edc46569817f95faa42dd4c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:49:13.973212 22497 raft_consensus.cc:504] T 2d91069a9a5f4eccad55af83cacfd1a6 P c1e154de2edc46569817f95faa42dd4c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1e154de2edc46569817f95faa42dd4c" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 34445 } } peers { permanent_uuid: "58b05d20c13e4055955cb7591d50f7b1" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 34513 } } peers { permanent_uuid: "f25bc24250484e5c9481ff5b2666f101" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 33957 } }
I20260807 08:49:13.973971 22497 leader_election.cc:288] T 2d91069a9a5f4eccad55af83cacfd1a6 P c1e154de2edc46569817f95faa42dd4c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 58b05d20c13e4055955cb7591d50f7b1 (127.4.159.129:34513), f25bc24250484e5c9481ff5b2666f101 (127.4.159.130:33957)
I20260807 08:49:13.977550 22463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:49:13.973710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.130:33957 (local address 127.4.159.131:47737)
0807 08:49:13.973795 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0807 08:49:13.973807 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 08:49:13.973845 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:49:13.974076 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:49:13.974078 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:49:13.974079 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:49:13.974142 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:49:13.974143 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:49:13.975150 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:49:13.975151 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:49:13.976334 (+ 1183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:49:13.976336 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:49:13.977240 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:49:13.977241 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:49:13.977352 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:49:13.977356 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 08:49:13.977406 (+ 50us) client_negotiation.cc:240] Negotiation successful
0807 08:49:13.977442 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68}
I20260807 08:49:13.977744 22478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:49:13.973882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.131:32855 (local address 127.4.159.129:34513)
0807 08:49:13.973952 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0807 08:49:13.973954 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:49:13.974198 (+ 244us) server_negotiation.cc:407] Connection header received
0807 08:49:13.974285 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:49:13.974286 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:49:13.974294 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:49:13.974319 (+ 25us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:49:13.974581 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:49:13.975224 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:49:13.976657 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:49:13.977362 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:49:13.977399 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:49:13.977406 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0807 08:49:13.977634 (+ 228us) server_negotiation.cc:299] Negotiation successful
0807 08:49:13.977681 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 08:49:13.977892 22479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:49:13.973909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.131:47737 (local address 127.4.159.130:33957)
0807 08:49:13.973948 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0807 08:49:13.973950 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:49:13.973957 (+ 7us) server_negotiation.cc:407] Connection header received
0807 08:49:13.973978 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:49:13.973979 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:49:13.973987 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:49:13.974032 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:49:13.974217 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:49:13.974965 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:49:13.976567 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:49:13.977123 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:49:13.977183 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:49:13.977193 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0807 08:49:13.977803 (+ 610us) server_negotiation.cc:299] Negotiation successful
0807 08:49:13.977840 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 08:49:13.978219 22160 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d91069a9a5f4eccad55af83cacfd1a6" candidate_uuid: "c1e154de2edc46569817f95faa42dd4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f25bc24250484e5c9481ff5b2666f101" is_pre_election: true
I20260807 08:49:13.978286 22499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:49:13.973712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.129:34513 (local address 127.4.159.131:32855)
0807 08:49:13.974145 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0807 08:49:13.974154 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 08:49:13.974220 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:49:13.974403 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:49:13.974404 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:49:13.974405 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:49:13.974505 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:49:13.974506 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:49:13.975302 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:49:13.975302 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:49:13.976576 (+ 1274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:49:13.976578 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:49:13.977457 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:49:13.977457 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:49:13.977549 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:49:13.977553 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 08:49:13.978103 (+ 550us) client_negotiation.cc:240] Negotiation successful
0807 08:49:13.978156 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":413,"mutex_wait_us":9,"thread_start_us":84,"threads_started":1}
I20260807 08:49:13.978387 22160 raft_consensus.cc:2454] T 2d91069a9a5f4eccad55af83cacfd1a6 P f25bc24250484e5c9481ff5b2666f101 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1e154de2edc46569817f95faa42dd4c in term 0.
I20260807 08:49:13.978664 22279 leader_election.cc:302] T 2d91069a9a5f4eccad55af83cacfd1a6 P c1e154de2edc46569817f95faa42dd4c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c1e154de2edc46569817f95faa42dd4c, f25bc24250484e5c9481ff5b2666f101; no voters:
I20260807 08:49:13.978629 21943 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d91069a9a5f4eccad55af83cacfd1a6" candidate_uuid: "c1e154de2edc46569817f95faa42dd4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58b05d20c13e4055955cb7591d50f7b1" is_pre_election: true
I20260807 08:49:13.978840 21943 raft_consensus.cc:2454] T 2d91069a9a5f4eccad55af83cacfd1a6 P 58b05d20c13e4055955cb7591d50f7b1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1e154de2edc46569817f95faa42dd4c in term 0.
I20260807 08:49:13.979014 22497 raft_consensus.cc:2781] T 2d91069a9a5f4eccad55af83cacfd1a6 P c1e154de2edc46569817f95faa42dd4c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:49:13.979066 22497 raft_consensus.cc:482] T 2d91069a9a5f4eccad55af83cacfd1a6 P c1e154de2edc46569817f95faa42dd4c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:49:13.979089 22497 raft_consensus.cc:3037] T 2d91069a9a5f4eccad55af83cacfd1a6 P c1e154de2edc46569817f95faa42dd4c [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:49:13.980019 22497 raft_consensus.cc:504] T 2d91069a9a5f4eccad55af83cacfd1a6 P c1e154de2edc46569817f95faa42dd4c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1e154de2edc46569817f95faa42dd4c" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 34445 } } peers { permanent_uuid: "58b05d20c13e4055955cb7591d50f7b1" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 34513 } } peers { permanent_uuid: "f25bc24250484e5c9481ff5b2666f101" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 33957 } }
I20260807 08:49:13.980141 22497 leader_election.cc:288] T 2d91069a9a5f4eccad55af83cacfd1a6 P c1e154de2edc46569817f95faa42dd4c [CANDIDATE]: Term 1 election: Requested vote from peers 58b05d20c13e4055955cb7591d50f7b1 (127.4.159.129:34513), f25bc24250484e5c9481ff5b2666f101 (127.4.159.130:33957)
I20260807 08:49:13.980472 21943 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d91069a9a5f4eccad55af83cacfd1a6" candidate_uuid: "c1e154de2edc46569817f95faa42dd4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58b05d20c13e4055955cb7591d50f7b1"
I20260807 08:49:13.980496 22160 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d91069a9a5f4eccad55af83cacfd1a6" candidate_uuid: "c1e154de2edc46569817f95faa42dd4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f25bc24250484e5c9481ff5b2666f101"
I20260807 08:49:13.980542 21943 raft_consensus.cc:3037] T 2d91069a9a5f4eccad55af83cacfd1a6 P 58b05d20c13e4055955cb7591d50f7b1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:49:13.980573 22160 raft_consensus.cc:3037] T 2d91069a9a5f4eccad55af83cacfd1a6 P f25bc24250484e5c9481ff5b2666f101 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:49:13.981572 21943 raft_consensus.cc:2454] T 2d91069a9a5f4eccad55af83cacfd1a6 P 58b05d20c13e4055955cb7591d50f7b1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1e154de2edc46569817f95faa42dd4c in term 1.
I20260807 08:49:13.981740 22160 raft_consensus.cc:2454] T 2d91069a9a5f4eccad55af83cacfd1a6 P f25bc24250484e5c9481ff5b2666f101 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1e154de2edc46569817f95faa42dd4c in term 1.
I20260807 08:49:13.981828 22280 leader_election.cc:302] T 2d91069a9a5f4eccad55af83cacfd1a6 P c1e154de2edc46569817f95faa42dd4c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58b05d20c13e4055955cb7591d50f7b1, c1e154de2edc46569817f95faa42dd4c; no voters:
I20260807 08:49:13.981938 22497 raft_consensus.cc:2781] T 2d91069a9a5f4eccad55af83cacfd1a6 P c1e154de2edc46569817f95faa42dd4c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:49:13.982079 22497 raft_consensus.cc:686] T 2d91069a9a5f4eccad55af83cacfd1a6 P c1e154de2edc46569817f95faa42dd4c [term 1 LEADER]: Becoming Leader. State: Replica: c1e154de2edc46569817f95faa42dd4c, State: Running, Role: LEADER
I20260807 08:49:13.982162 22497 consensus_queue.cc:237] T 2d91069a9a5f4eccad55af83cacfd1a6 P c1e154de2edc46569817f95faa42dd4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1e154de2edc46569817f95faa42dd4c" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 34445 } } peers { permanent_uuid: "58b05d20c13e4055955cb7591d50f7b1" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 34513 } } peers { permanent_uuid: "f25bc24250484e5c9481ff5b2666f101" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 33957 } }
I20260807 08:49:13.983075 21681 catalog_manager.cc:5495] T 2d91069a9a5f4eccad55af83cacfd1a6 P c1e154de2edc46569817f95faa42dd4c reported cstate change: term changed from 0 to 1, leader changed from <none> to c1e154de2edc46569817f95faa42dd4c (127.4.159.131). New cstate: current_term: 1 leader_uuid: "c1e154de2edc46569817f95faa42dd4c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1e154de2edc46569817f95faa42dd4c" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 34445 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58b05d20c13e4055955cb7591d50f7b1" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 34513 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f25bc24250484e5c9481ff5b2666f101" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 33957 } health_report { overall_health: UNKNOWN } } }
I20260807 08:49:14.000211 22482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:49:13.997109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58728 (local address 127.4.159.131:34445)
0807 08:49:13.997241 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0807 08:49:13.997243 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:49:13.997251 (+ 8us) server_negotiation.cc:407] Connection header received
0807 08:49:13.997279 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:49:13.997279 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:49:13.997297 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:49:13.997364 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:49:13.997526 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:49:13.998221 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:49:13.999281 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:49:13.999561 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:49:13.999645 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:49:13.999866 (+ 221us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:49:13.999982 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:49:14.000072 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0807 08:49:14.000124 (+ 52us) server_negotiation.cc:299] Negotiation successful
0807 08:49:14.000145 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110}
I20260807 08:49:14.002403 21943 raft_consensus.cc:1261] T 2d91069a9a5f4eccad55af83cacfd1a6 P 58b05d20c13e4055955cb7591d50f7b1 [term 1 FOLLOWER]: Refusing update from remote peer c1e154de2edc46569817f95faa42dd4c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:49:14.002425 22160 raft_consensus.cc:1261] T 2d91069a9a5f4eccad55af83cacfd1a6 P f25bc24250484e5c9481ff5b2666f101 [term 1 FOLLOWER]: Refusing update from remote peer c1e154de2edc46569817f95faa42dd4c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:49:14.002739 22497 consensus_queue.cc:1035] T 2d91069a9a5f4eccad55af83cacfd1a6 P c1e154de2edc46569817f95faa42dd4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "f25bc24250484e5c9481ff5b2666f101" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 33957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:49:14.002902 22497 consensus_queue.cc:1035] T 2d91069a9a5f4eccad55af83cacfd1a6 P c1e154de2edc46569817f95faa42dd4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "58b05d20c13e4055955cb7591d50f7b1" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 34513 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:49:14.008962 22505 mvcc.cc:205] Tried to move back new op lower bound from 7315835101191192576 to 7315835101111537664. Current Snapshot: MvccSnapshot[applied={T|T < 7315835101191192576}]
I20260807 08:49:14.008963 22507 mvcc.cc:205] Tried to move back new op lower bound from 7315835101191192576 to 7315835101111537664. Current Snapshot: MvccSnapshot[applied={T|T < 7315835101191192576}]
I20260807 08:49:14.009142 22504 mvcc.cc:205] Tried to move back new op lower bound from 7315835101191192576 to 7315835101111537664. Current Snapshot: MvccSnapshot[applied={T|T < 7315835101191192576}]
I20260807 08:49:14.012157 21677 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41714:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:49:14.012254 21677 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41714:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:49:14.012992 21677 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f0b0faa844f546198999b93ea74f18f5: Sending DeleteTablet for 3 replicas of tablet 2d91069a9a5f4eccad55af83cacfd1a6
I20260807 08:49:14.013477 21908 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d91069a9a5f4eccad55af83cacfd1a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:49:14 UTC) from {username='kudu', principal='kudu/127.4.159.190@KRBTEST.COM'} at 127.0.0.1:49978
I20260807 08:49:14.013716 22127 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d91069a9a5f4eccad55af83cacfd1a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:49:14 UTC) from {username='kudu', principal='kudu/127.4.159.190@KRBTEST.COM'} at 127.0.0.1:59840
I20260807 08:49:14.014060 22360 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d91069a9a5f4eccad55af83cacfd1a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:49:14 UTC) from {username='kudu', principal='kudu/127.4.159.190@KRBTEST.COM'} at 127.0.0.1:58714
I20260807 08:49:14.014732 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 21805
I20260807 08:49:14.030544 22516 tablet_replica.cc:330] T 2d91069a9a5f4eccad55af83cacfd1a6 P f25bc24250484e5c9481ff5b2666f101: stopping tablet replica
I20260807 08:49:14.030862 22516 raft_consensus.cc:2229] T 2d91069a9a5f4eccad55af83cacfd1a6 P f25bc24250484e5c9481ff5b2666f101 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:49:14.031050 22516 raft_consensus.cc:2258] T 2d91069a9a5f4eccad55af83cacfd1a6 P f25bc24250484e5c9481ff5b2666f101 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:49:14.031591 22516 ts_tablet_manager.cc:1905] T 2d91069a9a5f4eccad55af83cacfd1a6 P f25bc24250484e5c9481ff5b2666f101: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:49:14.031806 22517 tablet_replica.cc:330] T 2d91069a9a5f4eccad55af83cacfd1a6 P c1e154de2edc46569817f95faa42dd4c: stopping tablet replica
I20260807 08:49:14.033811 22516 ts_tablet_manager.cc:1918] T 2d91069a9a5f4eccad55af83cacfd1a6 P f25bc24250484e5c9481ff5b2666f101: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:49:14.033989 22517 raft_consensus.cc:2229] T 2d91069a9a5f4eccad55af83cacfd1a6 P c1e154de2edc46569817f95faa42dd4c [term 1 LEADER]: Raft consensus shutting down.
I20260807 08:49:14.034157 22517 raft_consensus.cc:2258] T 2d91069a9a5f4eccad55af83cacfd1a6 P c1e154de2edc46569817f95faa42dd4c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:49:14.034368 22516 log.cc:1196] T 2d91069a9a5f4eccad55af83cacfd1a6 P f25bc24250484e5c9481ff5b2666f101: Deleting WAL directory at /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/minicluster-data/ts-1/wal/wals/2d91069a9a5f4eccad55af83cacfd1a6
I20260807 08:49:14.034723 22517 ts_tablet_manager.cc:1905] T 2d91069a9a5f4eccad55af83cacfd1a6 P c1e154de2edc46569817f95faa42dd4c: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260807 08:49:14.035739 21659 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.159.129:34513 (error 108)
I20260807 08:49:14.036331 22516 ts_tablet_manager.cc:1939] T 2d91069a9a5f4eccad55af83cacfd1a6 P f25bc24250484e5c9481ff5b2666f101: Deleting consensus metadata
I20260807 08:49:14.036726 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 22027
I20260807 08:49:14.036761 22485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:49:14.036591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.129:34513 (local address 127.0.0.1:49992)
0807 08:49:14.036649 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0807 08:49:14.036707 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.129:34513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260807 08:49:14.036803 21658 catalog_manager.cc:4841] TS f25bc24250484e5c9481ff5b2666f101 (127.4.159.130:33957): tablet 2d91069a9a5f4eccad55af83cacfd1a6 (table test-table [id=14832a99fd2242248611bc1e9f4aed8e]) successfully deleted
W20260807 08:49:14.040866 21659 catalog_manager.cc:4584] TS 58b05d20c13e4055955cb7591d50f7b1 (127.4.159.129:34513): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2d91069a9a5f4eccad55af83cacfd1a6: Network error: Client connection negotiation failed: client connection to 127.4.159.129:34513: connect: Connection refused (error 111)
I20260807 08:49:14.045404 22517 ts_tablet_manager.cc:1918] T 2d91069a9a5f4eccad55af83cacfd1a6 P c1e154de2edc46569817f95faa42dd4c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:49:14.045521 22517 log.cc:1196] T 2d91069a9a5f4eccad55af83cacfd1a6 P c1e154de2edc46569817f95faa42dd4c: Deleting WAL directory at /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786092453486433-4734-0/minicluster-data/ts-2/wal/wals/2d91069a9a5f4eccad55af83cacfd1a6
I20260807 08:49:14.045820 22517 ts_tablet_manager.cc:1939] T 2d91069a9a5f4eccad55af83cacfd1a6 P c1e154de2edc46569817f95faa42dd4c: Deleting consensus metadata
I20260807 08:49:14.046290 21660 catalog_manager.cc:4841] TS c1e154de2edc46569817f95faa42dd4c (127.4.159.131:34445): tablet 2d91069a9a5f4eccad55af83cacfd1a6 (table test-table [id=14832a99fd2242248611bc1e9f4aed8e]) successfully deleted
I20260807 08:49:14.051847 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 22238
I20260807 08:49:14.071302 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 21630
I20260807 08:49:14.081056 4734 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260807 08:49:14.424619 4734 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260807 08:49:14.424721 4734 mini_ranger.cc:66] Stopping Ranger...
I20260807 08:49:14.768401 4734 mini_ranger.cc:68] Stopped Ranger
2026-08-07 08:49:14.768 UTC [21029] LOG: received smart shutdown request
2026-08-07 08:49:14.771 UTC [21029] LOG: background worker "logical replication launcher" (PID 21040) exited with exit code 1
2026-08-07 08:49:14.772 UTC [21033] LOG: shutting down
2026-08-07 08:49:14.897 UTC [21029] LOG: database system is shut down
2026-08-07T08:49:14Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (44985 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:49:15 dist-test-slave-6zbq krb5kdc[22536](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:49:15 dist-test-slave-6zbq krb5kdc[22536](info): set up 2 sockets
Aug 07 08:49:15 dist-test-slave-6zbq krb5kdc[22536](info): commencing operation
krb5kdc: starting...
W20260807 08:49:16.097564 4734 mini_kdc.cc:121] Time spent starting KDC: real 1.096s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/krb5kdc/test-user.keytab.
Aug 07 08:49:16 dist-test-slave-6zbq krb5kdc[22536](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092556, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:49:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:49:16Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.4.159.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.4.159.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.4.159.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/krb5kdc/rangeradmin_127.4.159.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.4.159.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/krb5kdc/rangeradmin_127.4.159.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.4.159.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.4.159.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.4.159.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/krb5kdc/rangerlookup_127.4.159.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.4.159.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/krb5kdc/rangerlookup_127.4.159.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.4.159.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.4.159.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.4.159.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/krb5kdc/HTTP_127.4.159.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.4.159.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/krb5kdc/HTTP_127.4.159.148@KRBTEST.COM.keytab.
I20260807 08:49:16.238130 4734 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskQz81E_/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/postgres -l logfile start
2026-08-07 08:49:18.313 UTC [22577] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-07 08:49:18.313 UTC [22577] LOG: listening on IPv4 address "127.4.159.148", port 60839
2026-08-07 08:49:18.316 UTC [22577] LOG: listening on Unix socket "/tmp/.s.PGSQL.60839"
2026-08-07 08:49:18.342 UTC [22580] LOG: database system was shut down at 2026-08-07 08:49:17 UTC
2026-08-07 08:49:18.345 UTC [22577] LOG: database system is ready to accept connections
I20260807 08:49:19.383769 4734 mini_postgres.cc:96] Postgres bound to 60839
2026-08-07 08:49:19.390 UTC [22595] FATAL: database "slave" does not exist
127.4.159.148:60839 - accepting connections
I20260807 08:49:19.390834 4734 mini_ranger.cc:161] Starting Ranger...
I20260807 08:49:19.404836 4734 mini_ranger.cc:84] Created miniranger Postgres user
I20260807 08:49:19.769273 4734 mini_ranger.cc:87] Created ranger Postgres database
I20260807 08:49:19.769351 4734 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ranger-admin
2026-08-07 08:49:20,024 [I] DB FLAVOR :POSTGRES
2026-08-07 08:49:20,024 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 08:49:20,024 [I] Checking connection..
2026-08-07 08:49:20,024 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQz81E_/build/release/bin/postgresql.jar:/tmp/dist-test-taskQz81E_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.159.148:60839/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-07 08:49:20,269 [I] Checking connection passed.
2026-08-07 08:49:20,270 [I] --------- Verifying version history table ---------
2026-08-07 08:49:20,270 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQz81E_/build/release/bin/postgresql.jar:/tmp/dist-test-taskQz81E_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.159.148:60839/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 08:49:20,510 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 08:49:20,510 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQz81E_/build/release/bin/postgresql.jar:/tmp/dist-test-taskQz81E_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.159.148:60839/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 08:49:20,745 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 08:49:20,745 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-07 08:49:20,745 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQz81E_/build/release/bin/postgresql.jar:/tmp/dist-test-taskQz81E_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.159.148:60839/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskQz81E_/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-07 08:49:20.993 UTC [22706] WARNING: there is no transaction in progress
2026-08-07 08:49:21,001 [I] create_dbversion_catalog.sql file imported successfully
2026-08-07 08:49:21,001 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQz81E_/build/release/bin/postgresql.jar:/tmp/dist-test-taskQz81E_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.159.148:60839/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 08:49:21,233 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-07 08:49:21,233 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 08:49:21,233 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQz81E_/build/release/bin/postgresql.jar:/tmp/dist-test-taskQz81E_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.159.148:60839/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-07 08:49:21,467 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQz81E_/build/release/bin/postgresql.jar:/tmp/dist-test-taskQz81E_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.159.148:60839/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-07 08:49:21,715 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQz81E_/build/release/bin/postgresql.jar:/tmp/dist-test-taskQz81E_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.159.148:60839/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-6zbq.c.gcp-upstream.internal','N') ;"
2026-08-07 08:49:21,941 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQz81E_/build/release/bin/postgresql.jar:/tmp/dist-test-taskQz81E_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.159.148:60839/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 08:49:22,193 [I] Table x_portal_user does not exist in database ranger
2026-08-07 08:49:22,194 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQz81E_/build/release/bin/postgresql.jar:/tmp/dist-test-taskQz81E_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.159.148:60839/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 08:49:22,448 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-07 08:49:22,448 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQz81E_/build/release/bin/postgresql.jar:/tmp/dist-test-taskQz81E_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.159.148:60839/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 08:49:22,696 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQz81E_/build/release/bin/postgresql.jar:/tmp/dist-test-taskQz81E_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.159.148:60839/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 08:49:22,934 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-07 08:49:22,934 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQz81E_/build/release/bin/postgresql.jar:/tmp/dist-test-taskQz81E_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.159.148:60839/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskQz81E_/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-07 08:49:23.794 UTC [22860] WARNING: there is no transaction in progress
2026-08-07 08:49:23.810 UTC [22860] WARNING: there is no transaction in progress
2026-08-07 08:49:23.824 UTC [22860] WARNING: there is no transaction in progress
2026-08-07 08:49:23.837 UTC [22860] WARNING: there is no transaction in progress
2026-08-07 08:49:23.854 UTC [22860] WARNING: there is no transaction in progress
2026-08-07 08:49:23.871 UTC [22860] WARNING: there is no transaction in progress
2026-08-07 08:49:23.921 UTC [22860] WARNING: there is no transaction in progress
2026-08-07 08:49:23.927 UTC [22860] WARNING: there is no transaction in progress
2026-08-07 08:49:23.940 UTC [22860] WARNING: there is no transaction in progress
2026-08-07 08:49:23.948 UTC [22860] WARNING: there is no transaction in progress
2026-08-07 08:49:23.957 UTC [22860] WARNING: there is no transaction in progress
2026-08-07 08:49:23.970 UTC [22860] WARNING: there is no transaction in progress
2026-08-07 08:49:23.979 UTC [22860] WARNING: there is no transaction in progress
2026-08-07 08:49:23.984 UTC [22860] WARNING: there is no transaction in progress
2026-08-07 08:49:23.989 UTC [22860] WARNING: there is no transaction in progress
2026-08-07 08:49:23.995 UTC [22860] WARNING: there is no transaction in progress
2026-08-07 08:49:24,519 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-07 08:49:24,519 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQz81E_/build/release/bin/postgresql.jar:/tmp/dist-test-taskQz81E_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.159.148:60839/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-07 08:49:24,752 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-07 08:49:24,752 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQz81E_/build/release/bin/postgresql.jar:/tmp/dist-test-taskQz81E_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.159.148:60839/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 08:49:25,004 [I] Table x_portal_user already exists in database 'ranger'
2026-08-07 08:49:25,004 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQz81E_/build/release/bin/postgresql.jar:/tmp/dist-test-taskQz81E_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.159.148:60839/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 08:49:25,233 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-07 08:49:25,234 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQz81E_/build/release/bin/postgresql.jar:/tmp/dist-test-taskQz81E_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.159.148:60839/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 08:49:25,486 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQz81E_/build/release/bin/postgresql.jar:/tmp/dist-test-taskQz81E_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.159.148:60839/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 08:49:25,713 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQz81E_/build/release/bin/postgresql.jar:/tmp/dist-test-taskQz81E_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.159.148:60839/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-6zbq.c.gcp-upstream.internal';"
2026-08-07 08:49:25,985 [I] CORE_DB_SCHEMA import status has been updated
2026-08-07 08:49:25,985 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQz81E_/build/release/bin/postgresql.jar:/tmp/dist-test-taskQz81E_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.159.148:60839/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-07 08:49:26,236 [I] DB_PATCHES have already been applied
I20260807 08:49:26.241421 4734 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ranger-admin:/tmp/dist-test-taskQz81E_/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskQz81E_/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskQz81E_/build/release/bin/postgresql.jar:/tmp/dist-test-taskQz81E_/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260807 08:49:26.241478 4734 mini_ranger.cc:193] Using host: 127.4.159.148
I20260807 08:49:26.245144 4734 mini_ranger.cc:227] Ranger admin URL: http://127.4.159.148:49965
Aug 07, 2026 8:49:26 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 8:49:26 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 8:49:26 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ranger-admin/ews/webapp
Aug 07, 2026 8:49:26 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 07, 2026 8:49:26 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 07, 2026 8:49:27 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ranger-admin/ews/webapp].
Aug 07, 2026 8:49:27 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260807 08:49:27.452283 4734 mini_ranger.cc:160] Time spent starting Ranger: real 8.061s user 0.000s sys 0.009s
Aug 07, 2026 8:49:46 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 07, 2026 8:49:46 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 07, 2026 8:49:46 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 07, 2026 8:49:46 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 07, 2026 8:49:46 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 07, 2026 8:49:46 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 07, 2026 8:49:46 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 07, 2026 8:49:46 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 07, 2026 8:49:46 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 07, 2026 8:49:46 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 07, 2026 8:49:46 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 07, 2026 8:49:46 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 07, 2026 8:49:46 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 07, 2026 8:49:46 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 07, 2026 8:49:46 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 07, 2026 8:49:46 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 07, 2026 8:49:46 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 07, 2026 8:49:47 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260807 08:49:48.241184 4734 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.4.159.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.4.159.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.4.159.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/krb5kdc/rangerkms_127.4.159.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.4.159.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/krb5kdc/rangerkms_127.4.159.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.4.159.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.4.159.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.4.159.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/krb5kdc/HTTP_127.4.159.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.4.159.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/krb5kdc/HTTP_127.4.159.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 07 08:49:48 dist-test-slave-6zbq krb5kdc[22536](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092588, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260807 08:49:48.335119 4734 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260807 08:49:48.343585 4734 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260807 08:49:49.011233 4734 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260807 08:49:49.011310 4734 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ranger-kms
I20260807 08:49:49.011328 4734 mini_ranger_kms.cc:227] Using postgres at 127.4.159.148:60839
2026-08-07 08:49:49,165 [I] DB FLAVOR :POSTGRES
2026-08-07 08:49:49,165 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 08:49:49,165 [I] Checking connection
2026-08-07 08:49:49,165 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQz81E_/build/release/bin/postgresql.jar:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.159.148:60839/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-07 08:49:49,388 [I] connection success
2026-08-07 08:49:49,388 [I] --------- Verifying Ranger DB tables ---------
2026-08-07 08:49:49,388 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-07 08:49:49,388 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQz81E_/build/release/bin/postgresql.jar:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.159.148:60839/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-07 08:49:49,610 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-07 08:49:49,610 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 08:49:49,610 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-07 08:49:49,610 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQz81E_/build/release/bin/postgresql.jar:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.159.148:60839/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-07 08:49:49,856 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260807 08:49:50.290784 4734 mini_ranger_kms.cc:310] Created kms service
I20260807 08:49:50.491336 4734 mini_ranger_kms.cc:326] Created kudu user
I20260807 08:49:50.533058 4734 mini_ranger_kms.cc:343] Created rangerkms user
Aug 07 08:49:50 dist-test-slave-6zbq krb5kdc[22536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092588, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.4.159.148@KRBTEST.COM
I20260807 08:49:50.906764 4734 mini_ranger_kms.cc:384] Added ranger policy
I20260807 08:49:50.906889 4734 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ranger-kms:/tmp/dist-test-taskQz81E_/build/release/bin/postgresql.jar:/tmp/dist-test-taskQz81E_/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskQz81E_/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260807 08:49:50.906908 4734 mini_ranger_kms.cc:240] Using host: 127.4.159.148
I20260807 08:49:50.910820 4734 mini_ranger_kms.cc:276] Ranger KMS PID: 23150
I20260807 08:49:50.910898 4734 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.4.159.148:38293
Aug 07, 2026 8:49:51 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 8:49:51 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 8:49:51 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 07, 2026 8:49:51 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 07, 2026 8:49:52 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ranger-kms/ews/webapp].
Aug 07, 2026 8:49:52 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 07 08:49:52 dist-test-slave-6zbq krb5kdc[22536](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786092592, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.4.159.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:49:53.324549 4734 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 4.989s user 0.004s sys 0.134s
I20260807 08:49:53.324656 4734 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260807 08:49:53.324702 4734 mini_ranger_kms.cc:401] 127.4.159.148:38293/kms/v1/keys
Aug 07 08:49:53 dist-test-slave-6zbq krb5kdc[22536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092592, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.4.159.148@KRBTEST.COM for HTTP/127.4.159.148@KRBTEST.COM
Aug 07 08:49:53 dist-test-slave-6zbq krb5kdc[22536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092592, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.4.159.148@KRBTEST.COM for HTTP/127.4.159.148@KRBTEST.COM
Aug 07 08:49:56 dist-test-slave-6zbq krb5kdc[22536](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092596, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.4.159.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.159.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.159.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab.
Entry for principal kudu/127.4.159.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab.
I20260807 08:49:56.059398 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.159.190:44421
--webserver_interface=127.4.159.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab
--principal=kudu/127.4.159.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:38993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.159.190:44421
--ranger_config_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.159.148:38293/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:49:56.148219 23224 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:49:56.148407 23224 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:49:56.148427 23224 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:49:56.149520 23224 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:49:56.149561 23224 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:49:56.149574 23224 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:49:56.149597 23224 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:49:56.149621 23224 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:49:56.151116 23224 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:38993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/master-0/wal
--ranger_kms_url=127.4.159.148:38293/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.4.159.190:44421
--ranger_config_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab
--principal=kudu/127.4.159.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.159.190:44421
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.159.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23224
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:49:56.151333 23224 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:49:56.151516 23224 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:49:56.156241 23230 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:49:56.156273 23236 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:49:56.156345 23224 server_base.cc:1034] running on GCE node
W20260807 08:49:56.156273 23229 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:49:56.156657 23224 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:49:56.156914 23224 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:49:56.158041 23224 hybrid_clock.cc:648] HybridClock initialized: now 1786092596158025 us; error 32 us; skew 500 ppm
Aug 07 08:49:56 dist-test-slave-6zbq krb5kdc[22536](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092596, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:49:56.163992 23224 init.cc:377] Logged in from keytab as kudu/127.4.159.190@KRBTEST.COM (short username kudu)
I20260807 08:49:56.164358 23224 webserver.cc:460] Webserver started at http://127.4.159.190:40313/ using document root <none> and password file <none>
I20260807 08:49:56.164544 23224 fs_manager.cc:359] Metadata directory not provided
I20260807 08:49:56.164582 23224 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:49:56.164695 23224 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 08:49:56 dist-test-slave-6zbq krb5kdc[22536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092596, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.190@KRBTEST.COM for HTTP/127.4.159.148@KRBTEST.COM
I20260807 08:49:56.217552 23224 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/master-0/data/instance:
uuid: "64b4667e647142b79a9ba6b789b99205"
format_stamp: "Formatted at 2026-08-07 08:49:56 on dist-test-slave-6zbq"
server_key: "847accf43a0d093ead019064e31ee2d2"
server_key_iv: "6beca155d3a7179e2229cbdc4f2d77cd"
server_key_version: "kuduclusterkey@0"
I20260807 08:49:56.217985 23224 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/master-0/wal/instance:
uuid: "64b4667e647142b79a9ba6b789b99205"
format_stamp: "Formatted at 2026-08-07 08:49:56 on dist-test-slave-6zbq"
server_key: "847accf43a0d093ead019064e31ee2d2"
server_key_iv: "6beca155d3a7179e2229cbdc4f2d77cd"
server_key_version: "kuduclusterkey@0"
I20260807 08:49:56.219743 23224 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260807 08:49:56.233527 23253 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:49:56.233700 23224 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 08:49:56.233851 23224 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/master-0/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/master-0/wal
uuid: "64b4667e647142b79a9ba6b789b99205"
format_stamp: "Formatted at 2026-08-07 08:49:56 on dist-test-slave-6zbq"
server_key: "847accf43a0d093ead019064e31ee2d2"
server_key_iv: "6beca155d3a7179e2229cbdc4f2d77cd"
server_key_version: "kuduclusterkey@0"
I20260807 08:49:56.233937 23224 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:49:56.280671 23224 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:49:56.282482 23224 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:49:56.282608 23224 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:49:56.286298 23224 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.190:44421
I20260807 08:49:56.286379 23326 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.190:44421 every 8 connection(s)
I20260807 08:49:56.286669 23224 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
I20260807 08:49:56.287266 23327 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:49:56.289346 23327 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64b4667e647142b79a9ba6b789b99205: Bootstrap starting.
I20260807 08:49:56.289844 23327 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 64b4667e647142b79a9ba6b789b99205: Neither blocks nor log segments found. Creating new log.
I20260807 08:49:56.290066 23327 log.cc:824] T 00000000000000000000000000000000 P 64b4667e647142b79a9ba6b789b99205: Log is configured to *not* fsync() on all Append() calls
I20260807 08:49:56.290674 23327 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64b4667e647142b79a9ba6b789b99205: No bootstrap required, opened a new log
I20260807 08:49:56.292559 23327 raft_consensus.cc:348] T 00000000000000000000000000000000 P 64b4667e647142b79a9ba6b789b99205 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64b4667e647142b79a9ba6b789b99205" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 44421 } }
I20260807 08:49:56.292693 23327 raft_consensus.cc:374] T 00000000000000000000000000000000 P 64b4667e647142b79a9ba6b789b99205 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:49:56.292722 23327 raft_consensus.cc:729] T 00000000000000000000000000000000 P 64b4667e647142b79a9ba6b789b99205 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64b4667e647142b79a9ba6b789b99205, State: Initialized, Role: FOLLOWER
I20260807 08:49:56.292821 23327 consensus_queue.cc:260] T 00000000000000000000000000000000 P 64b4667e647142b79a9ba6b789b99205 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64b4667e647142b79a9ba6b789b99205" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 44421 } }
I20260807 08:49:56.292870 23327 raft_consensus.cc:388] T 00000000000000000000000000000000 P 64b4667e647142b79a9ba6b789b99205 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:49:56.292889 23327 raft_consensus.cc:482] T 00000000000000000000000000000000 P 64b4667e647142b79a9ba6b789b99205 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:49:56.292912 23327 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 64b4667e647142b79a9ba6b789b99205 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:49:56.293386 23327 raft_consensus.cc:504] T 00000000000000000000000000000000 P 64b4667e647142b79a9ba6b789b99205 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64b4667e647142b79a9ba6b789b99205" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 44421 } }
I20260807 08:49:56.293465 23327 leader_election.cc:302] T 00000000000000000000000000000000 P 64b4667e647142b79a9ba6b789b99205 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 64b4667e647142b79a9ba6b789b99205; no voters:
I20260807 08:49:56.293574 23327 leader_election.cc:288] T 00000000000000000000000000000000 P 64b4667e647142b79a9ba6b789b99205 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:49:56.293612 23331 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 64b4667e647142b79a9ba6b789b99205 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:49:56.293778 23327 sys_catalog.cc:564] T 00000000000000000000000000000000 P 64b4667e647142b79a9ba6b789b99205 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:49:56.293788 23331 raft_consensus.cc:686] T 00000000000000000000000000000000 P 64b4667e647142b79a9ba6b789b99205 [term 1 LEADER]: Becoming Leader. State: Replica: 64b4667e647142b79a9ba6b789b99205, State: Running, Role: LEADER
I20260807 08:49:56.293938 23331 consensus_queue.cc:237] T 00000000000000000000000000000000 P 64b4667e647142b79a9ba6b789b99205 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64b4667e647142b79a9ba6b789b99205" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 44421 } }
I20260807 08:49:56.294286 23332 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64b4667e647142b79a9ba6b789b99205 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "64b4667e647142b79a9ba6b789b99205" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64b4667e647142b79a9ba6b789b99205" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 44421 } } }
I20260807 08:49:56.294390 23332 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64b4667e647142b79a9ba6b789b99205 [sys.catalog]: This master's current role is: LEADER
I20260807 08:49:56.294548 23333 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64b4667e647142b79a9ba6b789b99205 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 64b4667e647142b79a9ba6b789b99205. Latest consensus state: current_term: 1 leader_uuid: "64b4667e647142b79a9ba6b789b99205" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64b4667e647142b79a9ba6b789b99205" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 44421 } } }
I20260807 08:49:56.294823 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 23224
I20260807 08:49:56.294864 23333 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64b4667e647142b79a9ba6b789b99205 [sys.catalog]: This master's current role is: LEADER
I20260807 08:49:56.294917 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/master-0/wal/instance
I20260807 08:49:56.295097 4734 external_mini_cluster.cc:1428] Setting key ae50e6de10272314872bba4ec934c8f8
I20260807 08:49:56.294625 23341 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:49:56.295354 23327 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260807 08:49:56.295729 23341 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:49:56.300084 23341 catalog_manager.cc:1347] Generated new cluster ID: 214d7782124b44bb86daec4d9a64ad1b
I20260807 08:49:56.300138 23341 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 08:49:56 dist-test-slave-6zbq krb5kdc[22536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092596, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.159.190@KRBTEST.COM
I20260807 08:49:56.308832 23353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:49:56.295509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43282 (local address 127.4.159.190:44421)
0807 08:49:56.295683 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0807 08:49:56.295692 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 08:49:56.296889 (+ 1197us) server_negotiation.cc:407] Connection header received
0807 08:49:56.297592 (+ 703us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:49:56.297601 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:49:56.297710 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:49:56.297894 (+ 184us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:49:56.298650 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:49:56.300206 (+ 1556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:49:56.301327 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:49:56.302024 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:49:56.302109 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:49:56.304921 (+ 2812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:49:56.304925 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:49:56.304926 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:49:56.304938 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:49:56.306897 (+ 1959us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:49:56.307293 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:49:56.307296 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:49:56.307297 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:49:56.307388 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:49:56.307563 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:49:56.307564 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:49:56.307564 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:49:56.307859 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:49:56.307987 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0807 08:49:56.308486 (+ 499us) server_negotiation.cc:299] Negotiation successful
0807 08:49:56.308613 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":91,"threads_started":1}
I20260807 08:49:56.337646 23341 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:49:56.338315 23341 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:49:56.341779 23341 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 64b4667e647142b79a9ba6b789b99205: Generated new TSK 0
I20260807 08:49:56.341928 23341 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.159.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.159.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.159.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.129.keytab.
Entry for principal kudu/127.4.159.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.129.keytab.
I20260807 08:49:56.958176 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.159.148:38293/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.159.129:0
--local_ip_for_outbound_sockets=127.4.159.129
--webserver_interface=127.4.159.129
--webserver_port=0
--tserver_master_addrs=127.4.159.190:44421
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.129.keytab
--principal=kudu/127.4.159.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:38993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:49:57.049399 23397 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:49:57.049579 23397 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:49:57.049599 23397 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:49:57.050722 23397 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:49:57.050760 23397 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:49:57.050789 23397 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.159.129
I20260807 08:49:57.052376 23397 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:38993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.4.159.148:38293/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.129.keytab
--principal=kudu/127.4.159.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.159.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.159.129
--webserver_port=0
--tserver_master_addrs=127.4.159.190:44421
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23397
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.159.129
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:49:57.052559 23397 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:49:57.052748 23397 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:49:57.057910 23411 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:49:57.057917 23407 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:49:57.057942 23397 server_base.cc:1034] running on GCE node
W20260807 08:49:57.058159 23414 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:49:57.058382 23397 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:49:57.058655 23397 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:49:57.059839 23397 hybrid_clock.cc:648] HybridClock initialized: now 1786092597059824 us; error 26 us; skew 500 ppm
Aug 07 08:49:57 dist-test-slave-6zbq krb5kdc[22536](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092597, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:49:57.063230 23397 init.cc:377] Logged in from keytab as kudu/127.4.159.129@KRBTEST.COM (short username kudu)
I20260807 08:49:57.063763 23397 webserver.cc:460] Webserver started at http://127.4.159.129:34195/ using document root <none> and password file <none>
I20260807 08:49:57.064000 23397 fs_manager.cc:359] Metadata directory not provided
I20260807 08:49:57.064062 23397 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:49:57.064193 23397 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 08:49:57 dist-test-slave-6zbq krb5kdc[22536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092597, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.129@KRBTEST.COM for HTTP/127.4.159.148@KRBTEST.COM
I20260807 08:49:57.077399 23397 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ts-0/data/instance:
uuid: "9d9685c5fddf4b188be861d45570f4e9"
format_stamp: "Formatted at 2026-08-07 08:49:57 on dist-test-slave-6zbq"
server_key: "5019ebf622744d282f72f29eb14a8591"
server_key_iv: "860934281b246536a27e6c748b1cdb5f"
server_key_version: "kuduclusterkey@0"
I20260807 08:49:57.077870 23397 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ts-0/wal/instance:
uuid: "9d9685c5fddf4b188be861d45570f4e9"
format_stamp: "Formatted at 2026-08-07 08:49:57 on dist-test-slave-6zbq"
server_key: "5019ebf622744d282f72f29eb14a8591"
server_key_iv: "860934281b246536a27e6c748b1cdb5f"
server_key_version: "kuduclusterkey@0"
I20260807 08:49:57.082826 23397 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.000s
I20260807 08:49:57.094525 23422 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:49:57.094826 23397 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 08:49:57.094923 23397 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ts-0/wal
uuid: "9d9685c5fddf4b188be861d45570f4e9"
format_stamp: "Formatted at 2026-08-07 08:49:57 on dist-test-slave-6zbq"
server_key: "5019ebf622744d282f72f29eb14a8591"
server_key_iv: "860934281b246536a27e6c748b1cdb5f"
server_key_version: "kuduclusterkey@0"
I20260807 08:49:57.094998 23397 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:49:57.107628 23397 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:49:57.110149 23397 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:49:57.110404 23397 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:49:57.110841 23397 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:49:57.111261 23397 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:49:57.111337 23397 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:49:57.111394 23397 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:49:57.111433 23397 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:49:57.122859 23397 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.129:44725
I20260807 08:49:57.123351 23397 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ts-0/data/info.pb
I20260807 08:49:57.125614 23599 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.129:44725 every 8 connection(s)
I20260807 08:49:57.133077 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 23397
Aug 07 08:49:57 dist-test-slave-6zbq krb5kdc[22536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092597, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.129@KRBTEST.COM for kudu/127.4.159.190@KRBTEST.COM
I20260807 08:49:57.133176 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ts-0/wal/instance
I20260807 08:49:57.133369 4734 external_mini_cluster.cc:1428] Setting key 7a33c1dc085e67020558d8b49b60afbb
I20260807 08:49:57.136766 23602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:49:57.126268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.129:55715 (local address 127.4.159.190:44421)
0807 08:49:57.126485 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0807 08:49:57.126487 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:49:57.126754 (+ 267us) server_negotiation.cc:407] Connection header received
0807 08:49:57.127338 (+ 584us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:49:57.127339 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:49:57.127355 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:49:57.127463 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:49:57.128258 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:49:57.129037 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:49:57.130595 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:49:57.130906 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:49:57.130984 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:49:57.133596 (+ 2612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:49:57.133598 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:49:57.133598 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:49:57.133606 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:49:57.135334 (+ 1728us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:49:57.135820 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:49:57.135821 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:49:57.135822 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:49:57.135890 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:49:57.136154 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:49:57.136155 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:49:57.136155 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:49:57.136262 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:49:57.136323 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 08:49:57.136580 (+ 257us) server_negotiation.cc:299] Negotiation successful
0807 08:49:57.136672 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":113,"threads_started":1}
I20260807 08:49:57.136890 23603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:49:57.126242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:44421 (local address 127.4.159.129:55715)
0807 08:49:57.126616 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0807 08:49:57.126647 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0807 08:49:57.127233 (+ 586us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:49:57.127541 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:49:57.127544 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:49:57.127769 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:49:57.128133 (+ 364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:49:57.128141 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:49:57.129128 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:49:57.129142 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:49:57.130497 (+ 1355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:49:57.130501 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:49:57.131076 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:49:57.131077 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:49:57.131259 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:49:57.131924 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:49:57.131935 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:49:57.133487 (+ 1552us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:49:57.135422 (+ 1935us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:49:57.135429 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:49:57.135431 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:49:57.135719 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:49:57.135962 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:49:57.135963 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:49:57.135963 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:49:57.136062 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:49:57.136324 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:49:57.136331 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:49:57.136458 (+ 127us) client_negotiation.cc:769] Sending connection context
0807 08:49:57.136557 (+ 99us) client_negotiation.cc:240] Negotiation successful
0807 08:49:57.136689 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":208,"threads_started":1}
I20260807 08:49:57.137570 23600 heartbeater.cc:347] Connected to a master server at 127.4.159.190:44421
I20260807 08:49:57.137663 23600 heartbeater.cc:464] Registering TS with master...
I20260807 08:49:57.137831 23600 heartbeater.cc:511] Master 127.4.159.190:44421 requested a full tablet report, sending...
I20260807 08:49:57.138411 23276 ts_manager.cc:194] Registered new tserver with Master: 9d9685c5fddf4b188be861d45570f4e9 (127.4.159.129:44725)
I20260807 08:49:57.139307 23276 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.159.129@KRBTEST.COM'} at 127.4.159.129:55715
WARNING: no policy specified for kudu/127.4.159.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.159.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.159.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.130.keytab.
Entry for principal kudu/127.4.159.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.130.keytab.
I20260807 08:49:57.160559 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.159.148:38293/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.159.130:0
--local_ip_for_outbound_sockets=127.4.159.130
--webserver_interface=127.4.159.130
--webserver_port=0
--tserver_master_addrs=127.4.159.190:44421
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.130.keytab
--principal=kudu/127.4.159.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:38993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:49:57.307884 23611 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:49:57.308102 23611 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:49:57.308137 23611 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:49:57.309342 23611 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:49:57.309386 23611 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:49:57.309422 23611 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.159.130
I20260807 08:49:57.311022 23611 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:38993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.4.159.148:38293/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.130.keytab
--principal=kudu/127.4.159.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.159.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.159.130
--webserver_port=0
--tserver_master_addrs=127.4.159.190:44421
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23611
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.159.130
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:49:57.311259 23611 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:49:57.311514 23611 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:49:57.317668 23624 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:49:57.317900 23625 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:49:57.318132 23628 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:49:57.318904 23611 server_base.cc:1034] running on GCE node
I20260807 08:49:57.319162 23611 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:49:57.319418 23611 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:49:57.320567 23611 hybrid_clock.cc:648] HybridClock initialized: now 1786092597320552 us; error 31 us; skew 500 ppm
Aug 07 08:49:57 dist-test-slave-6zbq krb5kdc[22536](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092597, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:49:57.323560 23611 init.cc:377] Logged in from keytab as kudu/127.4.159.130@KRBTEST.COM (short username kudu)
I20260807 08:49:57.324272 23611 webserver.cc:460] Webserver started at http://127.4.159.130:44807/ using document root <none> and password file <none>
I20260807 08:49:57.324527 23611 fs_manager.cc:359] Metadata directory not provided
I20260807 08:49:57.324589 23611 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:49:57.324745 23611 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 08:49:57 dist-test-slave-6zbq krb5kdc[22536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092597, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.130@KRBTEST.COM for HTTP/127.4.159.148@KRBTEST.COM
I20260807 08:49:57.345156 23611 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ts-1/data/instance:
uuid: "3b8cb0c4f0b3470896fa34ba169b6540"
format_stamp: "Formatted at 2026-08-07 08:49:57 on dist-test-slave-6zbq"
server_key: "338bbd92ca5d7add4418b1dc0113f5e3"
server_key_iv: "14b68b1bcf829d7fa55802b256392fa4"
server_key_version: "kuduclusterkey@0"
I20260807 08:49:57.345685 23611 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ts-1/wal/instance:
uuid: "3b8cb0c4f0b3470896fa34ba169b6540"
format_stamp: "Formatted at 2026-08-07 08:49:57 on dist-test-slave-6zbq"
server_key: "338bbd92ca5d7add4418b1dc0113f5e3"
server_key_iv: "14b68b1bcf829d7fa55802b256392fa4"
server_key_version: "kuduclusterkey@0"
I20260807 08:49:57.350447 23611 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260807 08:49:57.365425 23639 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:49:57.365609 23611 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 08:49:57.365705 23611 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ts-1/wal
uuid: "3b8cb0c4f0b3470896fa34ba169b6540"
format_stamp: "Formatted at 2026-08-07 08:49:57 on dist-test-slave-6zbq"
server_key: "338bbd92ca5d7add4418b1dc0113f5e3"
server_key_iv: "14b68b1bcf829d7fa55802b256392fa4"
server_key_version: "kuduclusterkey@0"
I20260807 08:49:57.365795 23611 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:49:57.384093 23611 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:49:57.386472 23611 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:49:57.386659 23611 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:49:57.386914 23611 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:49:57.387321 23611 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:49:57.387403 23611 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:49:57.387460 23611 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:49:57.387507 23611 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 08:49:57 dist-test-slave-6zbq krb5kdc[22536](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786092597, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:49:57.401643 23611 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.130:42241
I20260807 08:49:57.402141 23611 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ts-1/data/info.pb
I20260807 08:49:57.410926 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 23611
I20260807 08:49:57.411015 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ts-1/wal/instance
I20260807 08:49:57.411204 4734 external_mini_cluster.cc:1428] Setting key 19a197b8e07750f76e329bf62b39dfc9
I20260807 08:49:57.414510 23821 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.130:42241 every 8 connection(s)
WARNING: no policy specified for kudu/127.4.159.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.159.131@KRBTEST.COM" created.
Aug 07 08:49:57 dist-test-slave-6zbq krb5kdc[22536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092597, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.130@KRBTEST.COM for kudu/127.4.159.190@KRBTEST.COM
I20260807 08:49:57.446017 23826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:49:57.415109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:44421 (local address 127.4.159.130:35241)
0807 08:49:57.418506 (+ 3397us) negotiation.cc:107] Waiting for socket to connect
0807 08:49:57.418555 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 08:49:57.419374 (+ 819us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:49:57.425767 (+ 6393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:49:57.425771 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:49:57.426007 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:49:57.426274 (+ 267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:49:57.426280 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:49:57.427339 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:49:57.427340 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:49:57.428580 (+ 1240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:49:57.428583 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:49:57.435522 (+ 6939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:49:57.435523 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:49:57.435726 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:49:57.438498 (+ 2772us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:49:57.438509 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:49:57.442645 (+ 4136us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:49:57.444395 (+ 1750us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:49:57.444402 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:49:57.444404 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:49:57.444693 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:49:57.444921 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:49:57.444923 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:49:57.444923 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:49:57.445051 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:49:57.445377 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:49:57.445383 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:49:57.445540 (+ 157us) client_negotiation.cc:769] Sending connection context
0807 08:49:57.445643 (+ 103us) client_negotiation.cc:240] Negotiation successful
0807 08:49:57.445790 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3317,"thread_start_us":88,"threads_started":1}
I20260807 08:49:57.446362 23602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:49:57.415095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.130:35241 (local address 127.4.159.190:44421)
0807 08:49:57.415144 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0807 08:49:57.415147 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:49:57.418695 (+ 3548us) server_negotiation.cc:407] Connection header received
0807 08:49:57.425510 (+ 6815us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:49:57.425512 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:49:57.425530 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:49:57.425562 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:49:57.426414 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:49:57.427224 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:49:57.435016 (+ 7792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:49:57.435332 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:49:57.435379 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:49:57.442767 (+ 7388us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:49:57.442768 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:49:57.442769 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:49:57.442777 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:49:57.444309 (+ 1532us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:49:57.444784 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:49:57.444786 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:49:57.444787 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:49:57.444852 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:49:57.445138 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:49:57.445139 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:49:57.445139 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:49:57.445286 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:49:57.445346 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0807 08:49:57.446153 (+ 807us) server_negotiation.cc:299] Negotiation successful
0807 08:49:57.446244 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 08:49:57.446731 23822 heartbeater.cc:347] Connected to a master server at 127.4.159.190:44421
I20260807 08:49:57.446835 23822 heartbeater.cc:464] Registering TS with master...
I20260807 08:49:57.447021 23822 heartbeater.cc:511] Master 127.4.159.190:44421 requested a full tablet report, sending...
I20260807 08:49:57.447415 23276 ts_manager.cc:194] Registered new tserver with Master: 3b8cb0c4f0b3470896fa34ba169b6540 (127.4.159.130:42241)
I20260807 08:49:57.447908 23276 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.159.130@KRBTEST.COM'} at 127.4.159.130:35241
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.159.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.131.keytab.
Entry for principal kudu/127.4.159.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.131.keytab.
I20260807 08:49:57.461256 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.159.148:38293/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.159.131:0
--local_ip_for_outbound_sockets=127.4.159.131
--webserver_interface=127.4.159.131
--webserver_port=0
--tserver_master_addrs=127.4.159.190:44421
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.131.keytab
--principal=kudu/127.4.159.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:38993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:49:57.604543 23831 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:49:57.604805 23831 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:49:57.604839 23831 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:49:57.606564 23831 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:49:57.606623 23831 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:49:57.606662 23831 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.159.131
I20260807 08:49:57.608817 23831 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:38993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.4.159.148:38293/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.131.keytab
--principal=kudu/127.4.159.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.159.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.159.131
--webserver_port=0
--tserver_master_addrs=127.4.159.190:44421
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23831
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.159.131
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:49:57.609045 23831 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:49:57.609427 23831 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 08:49:57.616922 23831 server_base.cc:1034] running on GCE node
W20260807 08:49:57.616930 23841 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:49:57.617144 23846 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:49:57.617108 23840 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:49:57.617424 23831 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:49:57.617664 23831 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:49:57.618810 23831 hybrid_clock.cc:648] HybridClock initialized: now 1786092597618795 us; error 32 us; skew 500 ppm
Aug 07 08:49:57 dist-test-slave-6zbq krb5kdc[22536](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092597, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:49:57.621528 23831 init.cc:377] Logged in from keytab as kudu/127.4.159.131@KRBTEST.COM (short username kudu)
I20260807 08:49:57.621949 23831 webserver.cc:460] Webserver started at http://127.4.159.131:36063/ using document root <none> and password file <none>
I20260807 08:49:57.622186 23831 fs_manager.cc:359] Metadata directory not provided
I20260807 08:49:57.622228 23831 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:49:57.622372 23831 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 08:49:57 dist-test-slave-6zbq krb5kdc[22536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092597, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.131@KRBTEST.COM for HTTP/127.4.159.148@KRBTEST.COM
I20260807 08:49:57.634965 23831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ts-2/data/instance:
uuid: "9c9cbb529f7e41a09d856763bc915e97"
format_stamp: "Formatted at 2026-08-07 08:49:57 on dist-test-slave-6zbq"
server_key: "6e394e47afed38dc73ee05b07ad5f4b6"
server_key_iv: "af6b92f22f48814fed8c7b18abfdae7c"
server_key_version: "kuduclusterkey@0"
I20260807 08:49:57.635383 23831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ts-2/wal/instance:
uuid: "9c9cbb529f7e41a09d856763bc915e97"
format_stamp: "Formatted at 2026-08-07 08:49:57 on dist-test-slave-6zbq"
server_key: "6e394e47afed38dc73ee05b07ad5f4b6"
server_key_iv: "af6b92f22f48814fed8c7b18abfdae7c"
server_key_version: "kuduclusterkey@0"
I20260807 08:49:57.637507 23831 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.001s
I20260807 08:49:57.658389 23857 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:49:57.662544 23831 fs_manager.cc:725] Time spent opening block manager: real 0.012s user 0.000s sys 0.000s
I20260807 08:49:57.662717 23831 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ts-2/wal
uuid: "9c9cbb529f7e41a09d856763bc915e97"
format_stamp: "Formatted at 2026-08-07 08:49:57 on dist-test-slave-6zbq"
server_key: "6e394e47afed38dc73ee05b07ad5f4b6"
server_key_iv: "af6b92f22f48814fed8c7b18abfdae7c"
server_key_version: "kuduclusterkey@0"
I20260807 08:49:57.662878 23831 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:49:57.686769 23831 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:49:57.688675 23831 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:49:57.688835 23831 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:49:57.689164 23831 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:49:57.689524 23831 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:49:57.689563 23831 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:49:57.689591 23831 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:49:57.689627 23831 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:49:57.701860 23831 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.131:37811
I20260807 08:49:57.701993 24022 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.131:37811 every 8 connection(s)
I20260807 08:49:57.702418 23831 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ts-2/data/info.pb
I20260807 08:49:57.704139 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 23831
I20260807 08:49:57.704252 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ts-2/wal/instance
I20260807 08:49:57.704469 4734 external_mini_cluster.cc:1428] Setting key 4413646d85c712f659c42f9a50ffde9c
Aug 07 08:49:57 dist-test-slave-6zbq krb5kdc[22536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092597, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.131@KRBTEST.COM for kudu/127.4.159.190@KRBTEST.COM
I20260807 08:49:57.711964 23602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:49:57.703429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.131:47071 (local address 127.4.159.190:44421)
0807 08:49:57.703481 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0807 08:49:57.703483 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:49:57.703709 (+ 226us) server_negotiation.cc:407] Connection header received
0807 08:49:57.704204 (+ 495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:49:57.704205 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:49:57.704218 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:49:57.704269 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:49:57.704948 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:49:57.705486 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:49:57.706691 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:49:57.706901 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:49:57.706955 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:49:57.709493 (+ 2538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:49:57.709494 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:49:57.709495 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:49:57.709503 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:49:57.710755 (+ 1252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:49:57.711116 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:49:57.711118 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:49:57.711118 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:49:57.711170 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:49:57.711385 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:49:57.711386 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:49:57.711386 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:49:57.711491 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:49:57.711547 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 08:49:57.711796 (+ 249us) server_negotiation.cc:299] Negotiation successful
0807 08:49:57.711876 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260807 08:49:57.712174 24026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:49:57.703369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:44421 (local address 127.4.159.131:47071)
0807 08:49:57.703602 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0807 08:49:57.703632 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0807 08:49:57.704105 (+ 473us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:49:57.704340 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:49:57.704344 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:49:57.704533 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:49:57.704870 (+ 337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:49:57.704879 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:49:57.705567 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:49:57.705568 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:49:57.706601 (+ 1033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:49:57.706604 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:49:57.707043 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:49:57.707044 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:49:57.707188 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:49:57.707787 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:49:57.707795 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:49:57.709409 (+ 1614us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:49:57.710825 (+ 1416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:49:57.710832 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:49:57.710834 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:49:57.711050 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:49:57.711222 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:49:57.711223 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:49:57.711223 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:49:57.711329 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:49:57.711545 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:49:57.711551 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:49:57.711679 (+ 128us) client_negotiation.cc:769] Sending connection context
0807 08:49:57.711772 (+ 93us) client_negotiation.cc:240] Negotiation successful
0807 08:49:57.711908 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":84,"threads_started":1}
I20260807 08:49:57.712684 24024 heartbeater.cc:347] Connected to a master server at 127.4.159.190:44421
I20260807 08:49:57.712774 24024 heartbeater.cc:464] Registering TS with master...
I20260807 08:49:57.712949 24024 heartbeater.cc:511] Master 127.4.159.190:44421 requested a full tablet report, sending...
I20260807 08:49:57.713343 23276 ts_manager.cc:194] Registered new tserver with Master: 9c9cbb529f7e41a09d856763bc915e97 (127.4.159.131:37811)
I20260807 08:49:57.713850 23276 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.159.131@KRBTEST.COM'} at 127.4.159.131:47071
I20260807 08:49:57.716197 4734 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 08:49:57.716802 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 23397
I20260807 08:49:57.743141 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 23611
I20260807 08:49:57.756675 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 23831
I20260807 08:49:57.777009 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 23224
I20260807 08:49:57.788810 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.159.190:44421
--webserver_interface=127.4.159.190
--webserver_port=40313
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab
--principal=kudu/127.4.159.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:38993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.159.190:44421
--ranger_config_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.159.148:38293/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.4.159.148:38293/kms with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:49:57.868064 24027 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:49:57.868249 24027 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:49:57.868268 24027 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:49:57.869311 24027 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:49:57.869341 24027 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:49:57.869354 24027 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:49:57.869364 24027 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:49:57.869381 24027 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:49:57.870899 24027 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:38993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.4.159.148:38293/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.4.159.190:44421
--ranger_config_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab
--principal=kudu/127.4.159.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.159.190:44421
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.159.190
--webserver_port=40313
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24027
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:49:57.871071 24027 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:49:57.871234 24027 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:49:57.875475 24042 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:49:57.875519 24040 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:49:57.875486 24039 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:49:57.875662 24027 server_base.cc:1034] running on GCE node
I20260807 08:49:57.875882 24027 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:49:57.876137 24027 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:49:57.877274 24027 hybrid_clock.cc:648] HybridClock initialized: now 1786092597877262 us; error 27 us; skew 500 ppm
Aug 07 08:49:57 dist-test-slave-6zbq krb5kdc[22536](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092597, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:49:57.879510 24027 init.cc:377] Logged in from keytab as kudu/127.4.159.190@KRBTEST.COM (short username kudu)
I20260807 08:49:57.879884 24027 webserver.cc:460] Webserver started at http://127.4.159.190:40313/ using document root <none> and password file <none>
I20260807 08:49:57.880079 24027 fs_manager.cc:359] Metadata directory not provided
I20260807 08:49:57.880131 24027 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 08:49:57 dist-test-slave-6zbq krb5kdc[22536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092597, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.190@KRBTEST.COM for HTTP/127.4.159.148@KRBTEST.COM
I20260807 08:49:57.930331 24027 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:49:57.931031 24053 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:49:57.931195 24027 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 08:49:57.931299 24027 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/master-0/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/master-0/wal
uuid: "64b4667e647142b79a9ba6b789b99205"
format_stamp: "Formatted at 2026-08-07 08:49:56 on dist-test-slave-6zbq"
server_key: "847accf43a0d093ead019064e31ee2d2"
server_key_iv: "6beca155d3a7179e2229cbdc4f2d77cd"
server_key_version: "kuduclusterkey@0"
I20260807 08:49:57.931624 24027 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:49:57.964856 24027 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:49:57.966300 24027 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:49:57.966460 24027 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:49:57.970165 24027 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.190:44421
I20260807 08:49:57.970227 24131 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.190:44421 every 8 connection(s)
I20260807 08:49:57.970582 24027 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
I20260807 08:49:57.971729 24133 sys_catalog.cc:263] Verifying existing consensus state
I20260807 08:49:57.972348 24133 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64b4667e647142b79a9ba6b789b99205: Bootstrap starting.
I20260807 08:49:57.974107 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 24027
I20260807 08:49:57.974368 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.159.148:38293/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.159.129:44725
--local_ip_for_outbound_sockets=127.4.159.129
--tserver_master_addrs=127.4.159.190:44421
--webserver_port=34195
--webserver_interface=127.4.159.129
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.129.keytab
--principal=kudu/127.4.159.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:38993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.4.159.148:38293/kms with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 08:49:57.979947 24133 log.cc:824] T 00000000000000000000000000000000 P 64b4667e647142b79a9ba6b789b99205: Log is configured to *not* fsync() on all Append() calls
I20260807 08:49:57.981446 24133 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64b4667e647142b79a9ba6b789b99205: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 08:49:57.981678 24133 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64b4667e647142b79a9ba6b789b99205: Bootstrap complete.
I20260807 08:49:57.983836 24133 raft_consensus.cc:348] T 00000000000000000000000000000000 P 64b4667e647142b79a9ba6b789b99205 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64b4667e647142b79a9ba6b789b99205" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 44421 } }
I20260807 08:49:57.984062 24133 raft_consensus.cc:729] T 00000000000000000000000000000000 P 64b4667e647142b79a9ba6b789b99205 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64b4667e647142b79a9ba6b789b99205, State: Initialized, Role: FOLLOWER
I20260807 08:49:57.984212 24133 consensus_queue.cc:260] T 00000000000000000000000000000000 P 64b4667e647142b79a9ba6b789b99205 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64b4667e647142b79a9ba6b789b99205" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 44421 } }
I20260807 08:49:57.984287 24133 raft_consensus.cc:388] T 00000000000000000000000000000000 P 64b4667e647142b79a9ba6b789b99205 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:49:57.984333 24133 raft_consensus.cc:482] T 00000000000000000000000000000000 P 64b4667e647142b79a9ba6b789b99205 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:49:57.984378 24133 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 64b4667e647142b79a9ba6b789b99205 [term 1 FOLLOWER]: Advancing to term 2
I20260807 08:49:57.985523 24133 raft_consensus.cc:504] T 00000000000000000000000000000000 P 64b4667e647142b79a9ba6b789b99205 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64b4667e647142b79a9ba6b789b99205" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 44421 } }
I20260807 08:49:57.985620 24133 leader_election.cc:302] T 00000000000000000000000000000000 P 64b4667e647142b79a9ba6b789b99205 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 64b4667e647142b79a9ba6b789b99205; no voters:
I20260807 08:49:57.985774 24133 leader_election.cc:288] T 00000000000000000000000000000000 P 64b4667e647142b79a9ba6b789b99205 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 08:49:57.985808 24137 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 64b4667e647142b79a9ba6b789b99205 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 08:49:57.985960 24137 raft_consensus.cc:686] T 00000000000000000000000000000000 P 64b4667e647142b79a9ba6b789b99205 [term 2 LEADER]: Becoming Leader. State: Replica: 64b4667e647142b79a9ba6b789b99205, State: Running, Role: LEADER
I20260807 08:49:57.986078 24133 sys_catalog.cc:564] T 00000000000000000000000000000000 P 64b4667e647142b79a9ba6b789b99205 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:49:57.986114 24137 consensus_queue.cc:237] T 00000000000000000000000000000000 P 64b4667e647142b79a9ba6b789b99205 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64b4667e647142b79a9ba6b789b99205" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 44421 } }
I20260807 08:49:57.986433 24137 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64b4667e647142b79a9ba6b789b99205 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 64b4667e647142b79a9ba6b789b99205. Latest consensus state: current_term: 2 leader_uuid: "64b4667e647142b79a9ba6b789b99205" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64b4667e647142b79a9ba6b789b99205" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 44421 } } }
I20260807 08:49:57.986505 24137 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64b4667e647142b79a9ba6b789b99205 [sys.catalog]: This master's current role is: LEADER
I20260807 08:49:57.986589 24138 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64b4667e647142b79a9ba6b789b99205 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "64b4667e647142b79a9ba6b789b99205" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64b4667e647142b79a9ba6b789b99205" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 44421 } } }
I20260807 08:49:57.986718 24143 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:49:57.987214 24143 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:49:57.987331 24143 catalog_manager.cc:1259] Loaded cluster ID: 214d7782124b44bb86daec4d9a64ad1b
I20260807 08:49:57.987356 24143 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:49:57.987596 24133 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260807 08:49:57.988531 24143 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:49:57.988689 24143 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 64b4667e647142b79a9ba6b789b99205: Loaded TSK: 0
I20260807 08:49:57.988873 24143 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:49:57.986720 24138 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64b4667e647142b79a9ba6b789b99205 [sys.catalog]: This master's current role is: LEADER
W20260807 08:49:58.083364 24135 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:49:58.083613 24135 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:49:58.083650 24135 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:49:58.085206 24135 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:49:58.085260 24135 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:49:58.085306 24135 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.159.129
I20260807 08:49:58.087563 24135 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:38993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.4.159.148:38293/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.129.keytab
--principal=kudu/127.4.159.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.159.129:44725
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.159.129
--webserver_port=34195
--tserver_master_addrs=127.4.159.190:44421
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24135
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.159.129
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:49:58.087905 24135 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:49:58.088156 24135 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:49:58.094367 24184 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:49:58.094688 24182 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:49:58.094955 24187 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:49:58.095942 24135 server_base.cc:1034] running on GCE node
I20260807 08:49:58.096256 24135 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:49:58.096526 24135 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:49:58.097693 24135 hybrid_clock.cc:648] HybridClock initialized: now 1786092598097657 us; error 49 us; skew 500 ppm
Aug 07 08:49:58 dist-test-slave-6zbq krb5kdc[22536](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092598, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:49:58.100749 24135 init.cc:377] Logged in from keytab as kudu/127.4.159.129@KRBTEST.COM (short username kudu)
I20260807 08:49:58.101241 24135 webserver.cc:460] Webserver started at http://127.4.159.129:34195/ using document root <none> and password file <none>
I20260807 08:49:58.101504 24135 fs_manager.cc:359] Metadata directory not provided
I20260807 08:49:58.101588 24135 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 08:49:58 dist-test-slave-6zbq krb5kdc[22536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092598, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.129@KRBTEST.COM for HTTP/127.4.159.148@KRBTEST.COM
I20260807 08:49:58.122804 24135 fs_manager.cc:709] Time spent opening directory manager: real 0.008s user 0.000s sys 0.001s
I20260807 08:49:58.126467 24196 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:49:58.126645 24135 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.001s
I20260807 08:49:58.126727 24135 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ts-0/wal
uuid: "9d9685c5fddf4b188be861d45570f4e9"
format_stamp: "Formatted at 2026-08-07 08:49:57 on dist-test-slave-6zbq"
server_key: "5019ebf622744d282f72f29eb14a8591"
server_key_iv: "860934281b246536a27e6c748b1cdb5f"
server_key_version: "kuduclusterkey@0"
I20260807 08:49:58.127904 24135 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:49:58.157363 24135 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:49:58.160306 24135 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:49:58.160470 24135 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:49:58.160753 24135 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:49:58.161221 24135 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:49:58.161334 24135 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:49:58.161387 24135 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:49:58.161405 24135 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:49:58.167984 24135 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.129:44725
I20260807 08:49:58.168418 24135 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ts-0/data/info.pb
I20260807 08:49:58.169481 24384 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.129:44725 every 8 connection(s)
I20260807 08:49:58.172102 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 24135
I20260807 08:49:58.172391 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.159.148:38293/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.159.130:42241
--local_ip_for_outbound_sockets=127.4.159.130
--tserver_master_addrs=127.4.159.190:44421
--webserver_port=44807
--webserver_interface=127.4.159.130
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.130.keytab
--principal=kudu/127.4.159.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:38993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.4.159.148:38293/kms with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 08:49:58 dist-test-slave-6zbq krb5kdc[22536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092598, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.129@KRBTEST.COM for kudu/127.4.159.190@KRBTEST.COM
I20260807 08:49:58.182394 24389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:49:58.170196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:44421 (local address 127.4.159.129:58545)
0807 08:49:58.170625 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0807 08:49:58.170670 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 08:49:58.171294 (+ 624us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:49:58.171803 (+ 509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:49:58.171807 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:49:58.172028 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:49:58.172297 (+ 269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:49:58.172304 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:49:58.173915 (+ 1611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:49:58.173916 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:49:58.175207 (+ 1291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:49:58.175210 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:49:58.176155 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:49:58.176157 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:49:58.176296 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:49:58.176817 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:49:58.176827 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:49:58.178327 (+ 1500us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:49:58.180496 (+ 2169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:49:58.180503 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:49:58.180505 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:49:58.180773 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:49:58.181105 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:49:58.181106 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:49:58.181106 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:49:58.181242 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:49:58.181736 (+ 494us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:49:58.181742 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:49:58.181891 (+ 149us) client_negotiation.cc:769] Sending connection context
0807 08:49:58.181990 (+ 99us) client_negotiation.cc:240] Negotiation successful
0807 08:49:58.182141 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":91,"threads_started":1}
I20260807 08:49:58.183035 24391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:49:58.170294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.129:58545 (local address 127.4.159.190:44421)
0807 08:49:58.170532 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0807 08:49:58.170541 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 08:49:58.170800 (+ 259us) server_negotiation.cc:407] Connection header received
0807 08:49:58.171529 (+ 729us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:49:58.171537 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:49:58.171598 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:49:58.171744 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:49:58.172432 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:49:58.173795 (+ 1363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:49:58.175322 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:49:58.175929 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:49:58.176045 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:49:58.178469 (+ 2424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:49:58.178473 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:49:58.178474 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:49:58.178485 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:49:58.180402 (+ 1917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:49:58.180887 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:49:58.180890 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:49:58.180890 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:49:58.181002 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:49:58.181334 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:49:58.181335 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:49:58.181336 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:49:58.181625 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:49:58.181706 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 08:49:58.182696 (+ 990us) server_negotiation.cc:299] Negotiation successful
0807 08:49:58.182832 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":115,"threads_started":1}
I20260807 08:49:58.183671 24385 heartbeater.cc:347] Connected to a master server at 127.4.159.190:44421
I20260807 08:49:58.183765 24385 heartbeater.cc:464] Registering TS with master...
I20260807 08:49:58.183936 24385 heartbeater.cc:511] Master 127.4.159.190:44421 requested a full tablet report, sending...
W20260807 08:49:58.184870 24385 heartbeater.cc:644] Failed to heartbeat to 127.4.159.190:44421 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 08:49:58.184943 24385 heartbeater.cc:464] Registering TS with master...
I20260807 08:49:58.185003 24385 heartbeater.cc:511] Master 127.4.159.190:44421 requested a full tablet report, sending...
I20260807 08:49:58.185371 24385 heartbeater.cc:464] Registering TS with master...
I20260807 08:49:58.185444 24385 heartbeater.cc:511] Master 127.4.159.190:44421 requested a full tablet report, sending...
W20260807 08:49:58.185834 24385 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 08:49:58.276767 24393 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:49:58.277010 24393 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:49:58.277046 24393 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:49:58.278604 24393 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:49:58.278649 24393 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:49:58.278674 24393 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.159.130
I20260807 08:49:58.280117 24393 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:38993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.4.159.148:38293/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.130.keytab
--principal=kudu/127.4.159.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.159.130:42241
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.159.130
--webserver_port=44807
--tserver_master_addrs=127.4.159.190:44421
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24393
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.159.130
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:49:58.280300 24393 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:49:58.280498 24393 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:49:58.285181 24398 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:49:58.285193 24401 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:49:58.285312 24399 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:49:58.285496 24393 server_base.cc:1034] running on GCE node
I20260807 08:49:58.285699 24393 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:49:58.285912 24393 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:49:58.287094 24393 hybrid_clock.cc:648] HybridClock initialized: now 1786092598287083 us; error 29 us; skew 500 ppm
Aug 07 08:49:58 dist-test-slave-6zbq krb5kdc[22536](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092598, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:49:58.289438 24393 init.cc:377] Logged in from keytab as kudu/127.4.159.130@KRBTEST.COM (short username kudu)
I20260807 08:49:58.289863 24393 webserver.cc:460] Webserver started at http://127.4.159.130:44807/ using document root <none> and password file <none>
I20260807 08:49:58.290061 24393 fs_manager.cc:359] Metadata directory not provided
I20260807 08:49:58.290122 24393 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 08:49:58 dist-test-slave-6zbq krb5kdc[22536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092598, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.130@KRBTEST.COM for HTTP/127.4.159.148@KRBTEST.COM
I20260807 08:49:58.307439 24393 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:49:58.308596 24413 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:49:58.308808 24393 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:49:58.308890 24393 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ts-1/wal
uuid: "3b8cb0c4f0b3470896fa34ba169b6540"
format_stamp: "Formatted at 2026-08-07 08:49:57 on dist-test-slave-6zbq"
server_key: "338bbd92ca5d7add4418b1dc0113f5e3"
server_key_iv: "14b68b1bcf829d7fa55802b256392fa4"
server_key_version: "kuduclusterkey@0"
I20260807 08:49:58.309401 24393 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:49:58.326293 24393 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:49:58.339099 24393 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:49:58.339303 24393 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:49:58.339586 24393 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:49:58.341009 24393 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:49:58.341056 24393 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:49:58.341100 24393 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:49:58.341141 24393 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:49:58.348048 24393 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.130:42241
I20260807 08:49:58.353063 24393 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ts-1/data/info.pb
I20260807 08:49:58.356294 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 24393
I20260807 08:49:58.356587 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.159.148:38293/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.159.131:37811
--local_ip_for_outbound_sockets=127.4.159.131
--tserver_master_addrs=127.4.159.190:44421
--webserver_port=36063
--webserver_interface=127.4.159.131
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.131.keytab
--principal=kudu/127.4.159.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:38993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.4.159.148:38293/kms with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 08:49:58.351524 24589 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.130:42241 every 8 connection(s)
Aug 07 08:49:58 dist-test-slave-6zbq krb5kdc[22536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092598, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.130@KRBTEST.COM for kudu/127.4.159.190@KRBTEST.COM
I20260807 08:49:58.368338 24391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:49:58.358457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.130:58715 (local address 127.4.159.190:44421)
0807 08:49:58.358515 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0807 08:49:58.358517 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:49:58.358529 (+ 12us) server_negotiation.cc:407] Connection header received
0807 08:49:58.358558 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:49:58.358559 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:49:58.358572 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:49:58.358652 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:49:58.359487 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:49:58.360256 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:49:58.361759 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:49:58.362063 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:49:58.362127 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:49:58.364735 (+ 2608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:49:58.364737 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:49:58.364737 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:49:58.364745 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:49:58.366771 (+ 2026us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:49:58.367265 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:49:58.367266 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:49:58.367267 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:49:58.367326 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:49:58.367616 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:49:58.367617 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:49:58.367617 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:49:58.367761 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:49:58.367806 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0807 08:49:58.368161 (+ 355us) server_negotiation.cc:299] Negotiation successful
0807 08:49:58.368249 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260807 08:49:58.369021 24593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:49:58.353991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:44421 (local address 127.4.159.130:58715)
0807 08:49:58.356201 (+ 2210us) negotiation.cc:107] Waiting for socket to connect
0807 08:49:58.356250 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 08:49:58.357144 (+ 894us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:49:58.358772 (+ 1628us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:49:58.358777 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:49:58.359112 (+ 335us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:49:58.359390 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:49:58.359398 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:49:58.360367 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:49:58.360368 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:49:58.361648 (+ 1280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:49:58.361651 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:49:58.362244 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:49:58.362246 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:49:58.362418 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:49:58.363047 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:49:58.363059 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:49:58.364621 (+ 1562us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:49:58.366879 (+ 2258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:49:58.366886 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:49:58.366888 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:49:58.367173 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:49:58.367400 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:49:58.367401 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:49:58.367401 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:49:58.367532 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:49:58.367852 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:49:58.367859 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:49:58.368006 (+ 147us) client_negotiation.cc:769] Sending connection context
0807 08:49:58.368658 (+ 652us) client_negotiation.cc:240] Negotiation successful
0807 08:49:58.368815 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2156,"thread_start_us":1274,"threads_started":1}
I20260807 08:49:58.369796 24590 heartbeater.cc:347] Connected to a master server at 127.4.159.190:44421
I20260807 08:49:58.369887 24590 heartbeater.cc:464] Registering TS with master...
I20260807 08:49:58.370071 24590 heartbeater.cc:511] Master 127.4.159.190:44421 requested a full tablet report, sending...
W20260807 08:49:58.370814 24590 heartbeater.cc:644] Failed to heartbeat to 127.4.159.190:44421 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 08:49:58.370882 24590 heartbeater.cc:464] Registering TS with master...
I20260807 08:49:58.370957 24590 heartbeater.cc:511] Master 127.4.159.190:44421 requested a full tablet report, sending...
I20260807 08:49:58.371333 24590 heartbeater.cc:464] Registering TS with master...
I20260807 08:49:58.371402 24590 heartbeater.cc:511] Master 127.4.159.190:44421 requested a full tablet report, sending...
W20260807 08:49:58.371724 24590 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 08:49:58.487747 24594 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:49:58.488090 24594 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:49:58.488204 24594 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:49:58.489851 24594 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:49:58.489951 24594 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:49:58.490039 24594 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.159.131
I20260807 08:49:58.492211 24594 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:38993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.4.159.148:38293/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.131.keytab
--principal=kudu/127.4.159.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.159.131:37811
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.159.131
--webserver_port=36063
--tserver_master_addrs=127.4.159.190:44421
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24594
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.159.131
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:49:58.492640 24594 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:49:58.492954 24594 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:49:58.499568 24608 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:49:58.499590 24606 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:49:58.499888 24594 server_base.cc:1034] running on GCE node
W20260807 08:49:58.499925 24604 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:49:58.500167 24594 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:49:58.500401 24594 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:49:58.501560 24594 hybrid_clock.cc:648] HybridClock initialized: now 1786092598501544 us; error 35 us; skew 500 ppm
Aug 07 08:49:58 dist-test-slave-6zbq krb5kdc[22536](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092598, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:49:58.504269 24594 init.cc:377] Logged in from keytab as kudu/127.4.159.131@KRBTEST.COM (short username kudu)
I20260807 08:49:58.504772 24594 webserver.cc:460] Webserver started at http://127.4.159.131:36063/ using document root <none> and password file <none>
I20260807 08:49:58.505014 24594 fs_manager.cc:359] Metadata directory not provided
I20260807 08:49:58.505091 24594 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 08:49:58 dist-test-slave-6zbq krb5kdc[22536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092598, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.131@KRBTEST.COM for HTTP/127.4.159.148@KRBTEST.COM
I20260807 08:49:58.521411 24594 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260807 08:49:58.522495 24618 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:49:58.522703 24594 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 08:49:58.522830 24594 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ts-2/wal
uuid: "9c9cbb529f7e41a09d856763bc915e97"
format_stamp: "Formatted at 2026-08-07 08:49:57 on dist-test-slave-6zbq"
server_key: "6e394e47afed38dc73ee05b07ad5f4b6"
server_key_iv: "af6b92f22f48814fed8c7b18abfdae7c"
server_key_version: "kuduclusterkey@0"
I20260807 08:49:58.523358 24594 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:49:58.553530 24594 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:49:58.555878 24594 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:49:58.556155 24594 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:49:58.556722 24594 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:49:58.557157 24594 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:49:58.557248 24594 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:49:58.557323 24594 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:49:58.557387 24594 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:49:58.578792 24594 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.131:37811
I20260807 08:49:58.579344 24594 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ts-2/data/info.pb
I20260807 08:49:58.586000 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 24594
I20260807 08:49:58.586098 24779 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.131:37811 every 8 connection(s)
I20260807 08:49:58.602941 24391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:49:58.586715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43284 (local address 127.4.159.190:44421)
0807 08:49:58.586764 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0807 08:49:58.586766 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:49:58.587894 (+ 1128us) server_negotiation.cc:407] Connection header received
0807 08:49:58.588025 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:49:58.588026 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:49:58.588040 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:49:58.588075 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:49:58.591249 (+ 3174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:49:58.591779 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:49:58.592816 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:49:58.593096 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:49:58.593151 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:49:58.600198 (+ 7047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:49:58.600200 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:49:58.600200 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:49:58.600207 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:49:58.601831 (+ 1624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:49:58.602196 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:49:58.602197 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:49:58.602198 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:49:58.602259 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:49:58.602443 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:49:58.602444 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:49:58.602444 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:49:58.602569 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:49:58.602739 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0807 08:49:58.602783 (+ 44us) server_negotiation.cc:299] Negotiation successful
0807 08:49:58.602850 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
Aug 07 08:49:58 dist-test-slave-6zbq krb5kdc[22536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092598, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.131@KRBTEST.COM for kudu/127.4.159.190@KRBTEST.COM
I20260807 08:49:58.626104 24784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:49:58.587408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.131:40057 (local address 127.4.159.190:44421)
0807 08:49:58.591359 (+ 3951us) server_negotiation.cc:206] Beginning negotiation
0807 08:49:58.591361 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:49:58.591368 (+ 7us) server_negotiation.cc:407] Connection header received
0807 08:49:58.592838 (+ 1470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:49:58.592840 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:49:58.592852 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:49:58.592946 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:49:58.594035 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:49:58.594789 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:49:58.597732 (+ 2943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:49:58.598048 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:49:58.598106 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:49:58.622956 (+ 24850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:49:58.622957 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:49:58.622958 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:49:58.622965 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:49:58.624425 (+ 1460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:49:58.625200 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:49:58.625201 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:49:58.625202 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:49:58.625257 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:49:58.625480 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:49:58.625480 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:49:58.625481 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:49:58.625587 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:49:58.625643 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 08:49:58.625909 (+ 266us) server_negotiation.cc:299] Negotiation successful
0807 08:49:58.625994 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3876,"thread_start_us":62,"threads_started":1}
I20260807 08:49:58.626457 24785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:49:58.587440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:44421 (local address 127.4.159.131:40057)
0807 08:49:58.590987 (+ 3547us) negotiation.cc:107] Waiting for socket to connect
0807 08:49:58.591036 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 08:49:58.592661 (+ 1625us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:49:58.593299 (+ 638us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:49:58.593304 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:49:58.593600 (+ 296us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:49:58.593924 (+ 324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:49:58.593931 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:49:58.594909 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:49:58.594910 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:49:58.597628 (+ 2718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:49:58.597632 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:49:58.599882 (+ 2250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:49:58.599884 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:49:58.600058 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:49:58.602393 (+ 2335us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:49:58.602403 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:49:58.622861 (+ 20458us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:49:58.624558 (+ 1697us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:49:58.624566 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:49:58.624569 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:49:58.625130 (+ 561us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:49:58.625313 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:49:58.625313 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:49:58.625314 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:49:58.625425 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:49:58.625643 (+ 218us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:49:58.625649 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:49:58.625789 (+ 140us) client_negotiation.cc:769] Sending connection context
0807 08:49:58.625880 (+ 91us) client_negotiation.cc:240] Negotiation successful
0807 08:49:58.626028 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3501,"thread_start_us":91,"threads_started":1}
I20260807 08:49:58.627094 24780 heartbeater.cc:347] Connected to a master server at 127.4.159.190:44421
I20260807 08:49:58.627190 24780 heartbeater.cc:464] Registering TS with master...
I20260807 08:49:58.627372 24780 heartbeater.cc:511] Master 127.4.159.190:44421 requested a full tablet report, sending...
W20260807 08:49:58.627950 24780 heartbeater.cc:644] Failed to heartbeat to 127.4.159.190:44421 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 08:49:58.628016 24780 heartbeater.cc:464] Registering TS with master...
I20260807 08:49:58.628082 24780 heartbeater.cc:511] Master 127.4.159.190:44421 requested a full tablet report, sending...
I20260807 08:49:58.628386 24780 heartbeater.cc:464] Registering TS with master...
I20260807 08:49:58.628461 24780 heartbeater.cc:511] Master 127.4.159.190:44421 requested a full tablet report, sending...
W20260807 08:49:58.628764 24780 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 08:49:59.185997 24385 heartbeater.cc:464] Registering TS with master...
I20260807 08:49:59.186136 24385 heartbeater.cc:511] Master 127.4.159.190:44421 requested a full tablet report, sending...
I20260807 08:49:59.186506 24081 ts_manager.cc:194] Registered new tserver with Master: 9d9685c5fddf4b188be861d45570f4e9 (127.4.159.129:44725)
I20260807 08:49:59.187186 24081 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.159.129@KRBTEST.COM'} at 127.4.159.129:58545
Aug 07 08:49:59 dist-test-slave-6zbq krb5kdc[22536](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786092599, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:49:59.371893 24590 heartbeater.cc:464] Registering TS with master...
I20260807 08:49:59.372030 24590 heartbeater.cc:511] Master 127.4.159.190:44421 requested a full tablet report, sending...
I20260807 08:49:59.372273 24081 ts_manager.cc:194] Registered new tserver with Master: 3b8cb0c4f0b3470896fa34ba169b6540 (127.4.159.130:42241)
I20260807 08:49:59.372664 24081 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.159.130@KRBTEST.COM'} at 127.4.159.130:58715
I20260807 08:49:59.628918 24780 heartbeater.cc:464] Registering TS with master...
I20260807 08:49:59.629070 24780 heartbeater.cc:511] Master 127.4.159.190:44421 requested a full tablet report, sending...
I20260807 08:49:59.629305 24081 ts_manager.cc:194] Registered new tserver with Master: 9c9cbb529f7e41a09d856763bc915e97 (127.4.159.131:37811)
I20260807 08:49:59.629695 24081 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.159.131@KRBTEST.COM'} at 127.4.159.131:40057
I20260807 08:49:59.639676 4734 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 08:49:59.648029 24813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:49:59.641095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43296 (local address 127.4.159.190:44421)
0807 08:49:59.641423 (+ 328us) server_negotiation.cc:206] Beginning negotiation
0807 08:49:59.641430 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 08:49:59.641645 (+ 215us) server_negotiation.cc:407] Connection header received
0807 08:49:59.641670 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:49:59.641671 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:49:59.641683 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:49:59.641740 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:49:59.642472 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:49:59.642959 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:49:59.643624 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:49:59.643817 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:49:59.643870 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:49:59.645405 (+ 1535us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:49:59.645407 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:49:59.645407 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:49:59.645415 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:49:59.646965 (+ 1550us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:49:59.647325 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:49:59.647326 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:49:59.647327 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:49:59.647382 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:49:59.647552 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:49:59.647553 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:49:59.647554 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:49:59.647669 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:49:59.647757 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0807 08:49:59.647860 (+ 103us) server_negotiation.cc:299] Negotiation successful
0807 08:49:59.647933 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":64,"threads_started":1}
I20260807 08:49:59.649755 24081 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43296:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 08:49:59.652135 24081 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:49:59.670367 24823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:49:59.660142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37286 (local address 127.4.159.131:37811)
0807 08:49:59.660512 (+ 370us) server_negotiation.cc:206] Beginning negotiation
0807 08:49:59.660517 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:49:59.662866 (+ 2349us) server_negotiation.cc:407] Connection header received
0807 08:49:59.666203 (+ 3337us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:49:59.666206 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:49:59.666225 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:49:59.666277 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:49:59.666518 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:49:59.667552 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:49:59.668845 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:49:59.669741 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:49:59.669838 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:49:59.669892 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 08:49:59.670197 (+ 305us) server_negotiation.cc:299] Negotiation successful
0807 08:49:59.670238 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":38,"threads_started":1}
I20260807 08:49:59.671336 24821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:49:59.660075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46128 (local address 127.4.159.130:42241)
0807 08:49:59.663206 (+ 3131us) server_negotiation.cc:206] Beginning negotiation
0807 08:49:59.663210 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:49:59.663219 (+ 9us) server_negotiation.cc:407] Connection header received
0807 08:49:59.663250 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:49:59.663252 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:49:59.663268 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:49:59.663332 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:49:59.663548 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:49:59.664596 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:49:59.666148 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:49:59.667170 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:49:59.667250 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:49:59.669608 (+ 2358us) server_negotiation.cc:1035] Waiting for connection context
0807 08:49:59.671208 (+ 1600us) server_negotiation.cc:299] Negotiation successful
0807 08:49:59.671258 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3098,"thread_start_us":61,"threads_started":1}
I20260807 08:49:59.671566 24824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:49:59.659862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.130:42241 (local address 127.0.0.1:46128)
0807 08:49:59.663045 (+ 3183us) negotiation.cc:107] Waiting for socket to connect
0807 08:49:59.663054 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 08:49:59.663092 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:49:59.663417 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:49:59.663419 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:49:59.663421 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:49:59.663480 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:49:59.663481 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:49:59.664749 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:49:59.664750 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:49:59.666050 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:49:59.666053 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:49:59.669644 (+ 3591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:49:59.669645 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:49:59.669731 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:49:59.669745 (+ 14us) client_negotiation.cc:769] Sending connection context
0807 08:49:59.669813 (+ 68us) client_negotiation.cc:240] Negotiation successful
0807 08:49:59.671492 (+ 1679us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3143,"thread_start_us":373,"threads_started":1}
I20260807 08:49:59.671816 24825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:49:59.659786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.131:37811 (local address 127.0.0.1:37286)
0807 08:49:59.662743 (+ 2957us) negotiation.cc:107] Waiting for socket to connect
0807 08:49:59.662785 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 08:49:59.662860 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:49:59.666403 (+ 3543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:49:59.666404 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:49:59.666406 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:49:59.666444 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:49:59.666445 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:49:59.667649 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:49:59.667650 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:49:59.668751 (+ 1101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:49:59.668754 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:49:59.669956 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:49:59.669956 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:49:59.670090 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:49:59.670094 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 08:49:59.670481 (+ 387us) client_negotiation.cc:240] Negotiation successful
0807 08:49:59.671599 (+ 1118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2901,"mutex_wait_us":1053,"thread_start_us":145,"threads_started":1}
I20260807 08:49:59.672374 24463 tablet_service.cc:1467] Processing CreateTablet for tablet 31dd65f1ab22460599fa81e8d522666f (DEFAULT_TABLE table=test-table [id=8be1d19ee257422d8e4331df268a0b77]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:49:59.672652 24659 tablet_service.cc:1467] Processing CreateTablet for tablet 31dd65f1ab22460599fa81e8d522666f (DEFAULT_TABLE table=test-table [id=8be1d19ee257422d8e4331df268a0b77]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:49:59.672799 24463 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31dd65f1ab22460599fa81e8d522666f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:49:59.672967 24659 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31dd65f1ab22460599fa81e8d522666f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:49:59.678496 24827 tablet_bootstrap.cc:492] T 31dd65f1ab22460599fa81e8d522666f P 9c9cbb529f7e41a09d856763bc915e97: Bootstrap starting.
I20260807 08:49:59.679486 24827 tablet_bootstrap.cc:654] T 31dd65f1ab22460599fa81e8d522666f P 9c9cbb529f7e41a09d856763bc915e97: Neither blocks nor log segments found. Creating new log.
I20260807 08:49:59.680181 24827 log.cc:824] T 31dd65f1ab22460599fa81e8d522666f P 9c9cbb529f7e41a09d856763bc915e97: Log is configured to *not* fsync() on all Append() calls
I20260807 08:49:59.680567 24829 tablet_bootstrap.cc:492] T 31dd65f1ab22460599fa81e8d522666f P 3b8cb0c4f0b3470896fa34ba169b6540: Bootstrap starting.
I20260807 08:49:59.681396 24829 tablet_bootstrap.cc:654] T 31dd65f1ab22460599fa81e8d522666f P 3b8cb0c4f0b3470896fa34ba169b6540: Neither blocks nor log segments found. Creating new log.
I20260807 08:49:59.682159 24829 log.cc:824] T 31dd65f1ab22460599fa81e8d522666f P 3b8cb0c4f0b3470896fa34ba169b6540: Log is configured to *not* fsync() on all Append() calls
I20260807 08:49:59.682385 24820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:49:59.659786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.129:44725 (local address 127.0.0.1:44164)
0807 08:49:59.670613 (+ 10827us) negotiation.cc:107] Waiting for socket to connect
0807 08:49:59.670623 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 08:49:59.670715 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:49:59.671028 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:49:59.671029 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:49:59.671031 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:49:59.677799 (+ 6768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:49:59.677802 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:49:59.679041 (+ 1239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:49:59.679042 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:49:59.680126 (+ 1084us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:49:59.680129 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:49:59.682137 (+ 2008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:49:59.682139 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:49:59.682199 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:49:59.682204 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 08:49:59.682256 (+ 52us) client_negotiation.cc:240] Negotiation successful
0807 08:49:59.682296 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10785,"mutex_wait_us":7136,"thread_start_us":60,"threads_started":1}
I20260807 08:49:59.682626 24826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:49:59.660668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44164 (local address 127.4.159.129:44725)
0807 08:49:59.660829 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0807 08:49:59.660833 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:49:59.670686 (+ 9853us) server_negotiation.cc:407] Connection header received
0807 08:49:59.670791 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:49:59.670793 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:49:59.670808 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:49:59.670857 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:49:59.677898 (+ 7041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:49:59.678937 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:49:59.680230 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:49:59.681978 (+ 1748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:49:59.682044 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:49:59.682100 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 08:49:59.682495 (+ 395us) server_negotiation.cc:299] Negotiation successful
0807 08:49:59.682536 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":87,"threads_started":1}
I20260807 08:49:59.683283 24276 tablet_service.cc:1467] Processing CreateTablet for tablet 31dd65f1ab22460599fa81e8d522666f (DEFAULT_TABLE table=test-table [id=8be1d19ee257422d8e4331df268a0b77]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:49:59.683544 24827 tablet_bootstrap.cc:492] T 31dd65f1ab22460599fa81e8d522666f P 9c9cbb529f7e41a09d856763bc915e97: No bootstrap required, opened a new log
I20260807 08:49:59.683555 24276 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31dd65f1ab22460599fa81e8d522666f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:49:59.683633 24827 ts_tablet_manager.cc:1397] T 31dd65f1ab22460599fa81e8d522666f P 9c9cbb529f7e41a09d856763bc915e97: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260807 08:49:59.686111 24834 tablet_bootstrap.cc:492] T 31dd65f1ab22460599fa81e8d522666f P 9d9685c5fddf4b188be861d45570f4e9: Bootstrap starting.
I20260807 08:49:59.687507 24827 raft_consensus.cc:348] T 31dd65f1ab22460599fa81e8d522666f P 9c9cbb529f7e41a09d856763bc915e97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c9cbb529f7e41a09d856763bc915e97" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 37811 } } peers { permanent_uuid: "9d9685c5fddf4b188be861d45570f4e9" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 44725 } } peers { permanent_uuid: "3b8cb0c4f0b3470896fa34ba169b6540" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 42241 } }
I20260807 08:49:59.687672 24827 raft_consensus.cc:374] T 31dd65f1ab22460599fa81e8d522666f P 9c9cbb529f7e41a09d856763bc915e97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:49:59.687703 24827 raft_consensus.cc:729] T 31dd65f1ab22460599fa81e8d522666f P 9c9cbb529f7e41a09d856763bc915e97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c9cbb529f7e41a09d856763bc915e97, State: Initialized, Role: FOLLOWER
I20260807 08:49:59.687814 24827 consensus_queue.cc:260] T 31dd65f1ab22460599fa81e8d522666f P 9c9cbb529f7e41a09d856763bc915e97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c9cbb529f7e41a09d856763bc915e97" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 37811 } } peers { permanent_uuid: "9d9685c5fddf4b188be861d45570f4e9" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 44725 } } peers { permanent_uuid: "3b8cb0c4f0b3470896fa34ba169b6540" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 42241 } }
I20260807 08:49:59.688108 24827 ts_tablet_manager.cc:1428] T 31dd65f1ab22460599fa81e8d522666f P 9c9cbb529f7e41a09d856763bc915e97: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20260807 08:49:59.688217 24780 heartbeater.cc:503] Master 127.4.159.190:44421 was elected leader, sending a full tablet report...
I20260807 08:49:59.692981 24834 tablet_bootstrap.cc:654] T 31dd65f1ab22460599fa81e8d522666f P 9d9685c5fddf4b188be861d45570f4e9: Neither blocks nor log segments found. Creating new log.
I20260807 08:49:59.693549 24834 log.cc:824] T 31dd65f1ab22460599fa81e8d522666f P 9d9685c5fddf4b188be861d45570f4e9: Log is configured to *not* fsync() on all Append() calls
I20260807 08:49:59.694459 24834 tablet_bootstrap.cc:492] T 31dd65f1ab22460599fa81e8d522666f P 9d9685c5fddf4b188be861d45570f4e9: No bootstrap required, opened a new log
I20260807 08:49:59.694526 24834 ts_tablet_manager.cc:1397] T 31dd65f1ab22460599fa81e8d522666f P 9d9685c5fddf4b188be861d45570f4e9: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20260807 08:49:59.694959 24829 tablet_bootstrap.cc:492] T 31dd65f1ab22460599fa81e8d522666f P 3b8cb0c4f0b3470896fa34ba169b6540: No bootstrap required, opened a new log
I20260807 08:49:59.695030 24829 ts_tablet_manager.cc:1397] T 31dd65f1ab22460599fa81e8d522666f P 3b8cb0c4f0b3470896fa34ba169b6540: Time spent bootstrapping tablet: real 0.015s user 0.001s sys 0.000s
I20260807 08:49:59.698405 24829 raft_consensus.cc:348] T 31dd65f1ab22460599fa81e8d522666f P 3b8cb0c4f0b3470896fa34ba169b6540 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c9cbb529f7e41a09d856763bc915e97" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 37811 } } peers { permanent_uuid: "9d9685c5fddf4b188be861d45570f4e9" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 44725 } } peers { permanent_uuid: "3b8cb0c4f0b3470896fa34ba169b6540" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 42241 } }
I20260807 08:49:59.698561 24829 raft_consensus.cc:374] T 31dd65f1ab22460599fa81e8d522666f P 3b8cb0c4f0b3470896fa34ba169b6540 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:49:59.698591 24829 raft_consensus.cc:729] T 31dd65f1ab22460599fa81e8d522666f P 3b8cb0c4f0b3470896fa34ba169b6540 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b8cb0c4f0b3470896fa34ba169b6540, State: Initialized, Role: FOLLOWER
I20260807 08:49:59.698696 24829 consensus_queue.cc:260] T 31dd65f1ab22460599fa81e8d522666f P 3b8cb0c4f0b3470896fa34ba169b6540 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c9cbb529f7e41a09d856763bc915e97" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 37811 } } peers { permanent_uuid: "9d9685c5fddf4b188be861d45570f4e9" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 44725 } } peers { permanent_uuid: "3b8cb0c4f0b3470896fa34ba169b6540" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 42241 } }
I20260807 08:49:59.698930 24829 ts_tablet_manager.cc:1428] T 31dd65f1ab22460599fa81e8d522666f P 3b8cb0c4f0b3470896fa34ba169b6540: Time spent starting tablet: real 0.004s user 0.002s sys 0.003s
I20260807 08:49:59.698985 24590 heartbeater.cc:503] Master 127.4.159.190:44421 was elected leader, sending a full tablet report...
I20260807 08:49:59.703501 24834 raft_consensus.cc:348] T 31dd65f1ab22460599fa81e8d522666f P 9d9685c5fddf4b188be861d45570f4e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c9cbb529f7e41a09d856763bc915e97" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 37811 } } peers { permanent_uuid: "9d9685c5fddf4b188be861d45570f4e9" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 44725 } } peers { permanent_uuid: "3b8cb0c4f0b3470896fa34ba169b6540" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 42241 } }
I20260807 08:49:59.703644 24834 raft_consensus.cc:374] T 31dd65f1ab22460599fa81e8d522666f P 9d9685c5fddf4b188be861d45570f4e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:49:59.703673 24834 raft_consensus.cc:729] T 31dd65f1ab22460599fa81e8d522666f P 9d9685c5fddf4b188be861d45570f4e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d9685c5fddf4b188be861d45570f4e9, State: Initialized, Role: FOLLOWER
I20260807 08:49:59.703794 24834 consensus_queue.cc:260] T 31dd65f1ab22460599fa81e8d522666f P 9d9685c5fddf4b188be861d45570f4e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c9cbb529f7e41a09d856763bc915e97" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 37811 } } peers { permanent_uuid: "9d9685c5fddf4b188be861d45570f4e9" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 44725 } } peers { permanent_uuid: "3b8cb0c4f0b3470896fa34ba169b6540" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 42241 } }
I20260807 08:49:59.704027 24834 ts_tablet_manager.cc:1428] T 31dd65f1ab22460599fa81e8d522666f P 9d9685c5fddf4b188be861d45570f4e9: Time spent starting tablet: real 0.009s user 0.001s sys 0.003s
I20260807 08:49:59.705801 24385 heartbeater.cc:503] Master 127.4.159.190:44421 was elected leader, sending a full tablet report...
Aug 07 08:49:59 dist-test-slave-6zbq krb5kdc[22536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092599, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.190@KRBTEST.COM for HTTP/127.4.159.148@KRBTEST.COM
W20260807 08:49:59.837713 24782 tablet.cc:2369] T 31dd65f1ab22460599fa81e8d522666f P 9c9cbb529f7e41a09d856763bc915e97: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 08:49:59.860217 24591 tablet.cc:2369] T 31dd65f1ab22460599fa81e8d522666f P 3b8cb0c4f0b3470896fa34ba169b6540: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 08:49:59.862309 24838 raft_consensus.cc:482] T 31dd65f1ab22460599fa81e8d522666f P 3b8cb0c4f0b3470896fa34ba169b6540 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:49:59.862502 24838 raft_consensus.cc:504] T 31dd65f1ab22460599fa81e8d522666f P 3b8cb0c4f0b3470896fa34ba169b6540 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c9cbb529f7e41a09d856763bc915e97" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 37811 } } peers { permanent_uuid: "9d9685c5fddf4b188be861d45570f4e9" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 44725 } } peers { permanent_uuid: "3b8cb0c4f0b3470896fa34ba169b6540" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 42241 } }
I20260807 08:49:59.863118 24838 leader_election.cc:288] T 31dd65f1ab22460599fa81e8d522666f P 3b8cb0c4f0b3470896fa34ba169b6540 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9c9cbb529f7e41a09d856763bc915e97 (127.4.159.131:37811), 9d9685c5fddf4b188be861d45570f4e9 (127.4.159.129:44725)
I20260807 08:49:59.867552 24826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:49:59.863578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.130:51791 (local address 127.4.159.129:44725)
0807 08:49:59.863624 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0807 08:49:59.863627 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:49:59.863821 (+ 194us) server_negotiation.cc:407] Connection header received
0807 08:49:59.863931 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:49:59.863932 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:49:59.863944 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:49:59.863976 (+ 32us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:49:59.864228 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:49:59.864899 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:49:59.866518 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:49:59.867143 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:49:59.867188 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:49:59.867196 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0807 08:49:59.867450 (+ 254us) server_negotiation.cc:299] Negotiation successful
0807 08:49:59.867485 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 08:49:59.867552 24823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:49:59.863580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.130:49571 (local address 127.4.159.131:37811)
0807 08:49:59.863627 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0807 08:49:59.863629 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:49:59.863756 (+ 127us) server_negotiation.cc:407] Connection header received
0807 08:49:59.863869 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:49:59.863870 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:49:59.863880 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:49:59.863911 (+ 31us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:49:59.864151 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:49:59.864878 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:49:59.866519 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:49:59.867137 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:49:59.867188 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:49:59.867196 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0807 08:49:59.867437 (+ 241us) server_negotiation.cc:299] Negotiation successful
0807 08:49:59.867481 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 08:49:59.867811 24840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:49:59.863288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.131:37811 (local address 127.4.159.130:49571)
0807 08:49:59.863689 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0807 08:49:59.863700 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 08:49:59.863786 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:49:59.863985 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:49:59.863985 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:49:59.863987 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:49:59.864070 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:49:59.864072 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:49:59.864970 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:49:59.864971 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:49:59.866414 (+ 1443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:49:59.866417 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:49:59.867280 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:49:59.867281 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:49:59.867349 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:49:59.867354 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 08:49:59.867674 (+ 320us) client_negotiation.cc:240] Negotiation successful
0807 08:49:59.867722 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":147,"threads_started":1}
I20260807 08:49:59.868129 24841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:49:59.863339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.129:44725 (local address 127.4.159.130:51791)
0807 08:49:59.863753 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0807 08:49:59.863764 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 08:49:59.863850 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:49:59.864049 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:49:59.864050 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:49:59.864052 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:49:59.864142 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:49:59.864144 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:49:59.864980 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:49:59.864981 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:49:59.866414 (+ 1433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:49:59.866417 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:49:59.867305 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:49:59.867306 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:49:59.867374 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:49:59.867376 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 08:49:59.867997 (+ 621us) client_negotiation.cc:240] Negotiation successful
0807 08:49:59.868041 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":381,"mutex_wait_us":9,"thread_start_us":130,"threads_started":1}
I20260807 08:49:59.868243 24713 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31dd65f1ab22460599fa81e8d522666f" candidate_uuid: "3b8cb0c4f0b3470896fa34ba169b6540" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c9cbb529f7e41a09d856763bc915e97" is_pre_election: true
I20260807 08:49:59.868397 24713 raft_consensus.cc:2454] T 31dd65f1ab22460599fa81e8d522666f P 9c9cbb529f7e41a09d856763bc915e97 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b8cb0c4f0b3470896fa34ba169b6540 in term 0.
I20260807 08:49:59.868713 24415 leader_election.cc:302] T 31dd65f1ab22460599fa81e8d522666f P 3b8cb0c4f0b3470896fa34ba169b6540 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b8cb0c4f0b3470896fa34ba169b6540, 9c9cbb529f7e41a09d856763bc915e97; no voters:
I20260807 08:49:59.868705 24309 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31dd65f1ab22460599fa81e8d522666f" candidate_uuid: "3b8cb0c4f0b3470896fa34ba169b6540" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d9685c5fddf4b188be861d45570f4e9" is_pre_election: true
I20260807 08:49:59.868829 24838 raft_consensus.cc:2781] T 31dd65f1ab22460599fa81e8d522666f P 3b8cb0c4f0b3470896fa34ba169b6540 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:49:59.868845 24309 raft_consensus.cc:2454] T 31dd65f1ab22460599fa81e8d522666f P 9d9685c5fddf4b188be861d45570f4e9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b8cb0c4f0b3470896fa34ba169b6540 in term 0.
I20260807 08:49:59.868870 24838 raft_consensus.cc:482] T 31dd65f1ab22460599fa81e8d522666f P 3b8cb0c4f0b3470896fa34ba169b6540 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:49:59.868892 24838 raft_consensus.cc:3037] T 31dd65f1ab22460599fa81e8d522666f P 3b8cb0c4f0b3470896fa34ba169b6540 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:49:59.869792 24838 raft_consensus.cc:504] T 31dd65f1ab22460599fa81e8d522666f P 3b8cb0c4f0b3470896fa34ba169b6540 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c9cbb529f7e41a09d856763bc915e97" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 37811 } } peers { permanent_uuid: "9d9685c5fddf4b188be861d45570f4e9" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 44725 } } peers { permanent_uuid: "3b8cb0c4f0b3470896fa34ba169b6540" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 42241 } }
I20260807 08:49:59.869905 24838 leader_election.cc:288] T 31dd65f1ab22460599fa81e8d522666f P 3b8cb0c4f0b3470896fa34ba169b6540 [CANDIDATE]: Term 1 election: Requested vote from peers 9c9cbb529f7e41a09d856763bc915e97 (127.4.159.131:37811), 9d9685c5fddf4b188be861d45570f4e9 (127.4.159.129:44725)
I20260807 08:49:59.870101 24713 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31dd65f1ab22460599fa81e8d522666f" candidate_uuid: "3b8cb0c4f0b3470896fa34ba169b6540" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c9cbb529f7e41a09d856763bc915e97"
I20260807 08:49:59.870093 24309 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31dd65f1ab22460599fa81e8d522666f" candidate_uuid: "3b8cb0c4f0b3470896fa34ba169b6540" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d9685c5fddf4b188be861d45570f4e9"
I20260807 08:49:59.870177 24309 raft_consensus.cc:3037] T 31dd65f1ab22460599fa81e8d522666f P 9d9685c5fddf4b188be861d45570f4e9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:49:59.870177 24713 raft_consensus.cc:3037] T 31dd65f1ab22460599fa81e8d522666f P 9c9cbb529f7e41a09d856763bc915e97 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:49:59.871222 24713 raft_consensus.cc:2454] T 31dd65f1ab22460599fa81e8d522666f P 9c9cbb529f7e41a09d856763bc915e97 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b8cb0c4f0b3470896fa34ba169b6540 in term 1.
I20260807 08:49:59.871261 24309 raft_consensus.cc:2454] T 31dd65f1ab22460599fa81e8d522666f P 9d9685c5fddf4b188be861d45570f4e9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b8cb0c4f0b3470896fa34ba169b6540 in term 1.
I20260807 08:49:59.871454 24418 leader_election.cc:302] T 31dd65f1ab22460599fa81e8d522666f P 3b8cb0c4f0b3470896fa34ba169b6540 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b8cb0c4f0b3470896fa34ba169b6540, 9d9685c5fddf4b188be861d45570f4e9; no voters:
I20260807 08:49:59.871572 24838 raft_consensus.cc:2781] T 31dd65f1ab22460599fa81e8d522666f P 3b8cb0c4f0b3470896fa34ba169b6540 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:49:59.871711 24838 raft_consensus.cc:686] T 31dd65f1ab22460599fa81e8d522666f P 3b8cb0c4f0b3470896fa34ba169b6540 [term 1 LEADER]: Becoming Leader. State: Replica: 3b8cb0c4f0b3470896fa34ba169b6540, State: Running, Role: LEADER
I20260807 08:49:59.871802 24838 consensus_queue.cc:237] T 31dd65f1ab22460599fa81e8d522666f P 3b8cb0c4f0b3470896fa34ba169b6540 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c9cbb529f7e41a09d856763bc915e97" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 37811 } } peers { permanent_uuid: "9d9685c5fddf4b188be861d45570f4e9" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 44725 } } peers { permanent_uuid: "3b8cb0c4f0b3470896fa34ba169b6540" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 42241 } }
I20260807 08:49:59.872519 24081 catalog_manager.cc:5495] T 31dd65f1ab22460599fa81e8d522666f P 3b8cb0c4f0b3470896fa34ba169b6540 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3b8cb0c4f0b3470896fa34ba169b6540 (127.4.159.130). New cstate: current_term: 1 leader_uuid: "3b8cb0c4f0b3470896fa34ba169b6540" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c9cbb529f7e41a09d856763bc915e97" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 37811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d9685c5fddf4b188be861d45570f4e9" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 44725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b8cb0c4f0b3470896fa34ba169b6540" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 42241 } health_report { overall_health: HEALTHY } } }
I20260807 08:49:59.898702 24821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:49:59.896480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46130 (local address 127.4.159.130:42241)
0807 08:49:59.896526 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0807 08:49:59.896528 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:49:59.896535 (+ 7us) server_negotiation.cc:407] Connection header received
0807 08:49:59.896562 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:49:59.896563 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:49:59.896579 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:49:59.896636 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:49:59.896760 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:49:59.897270 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:49:59.897992 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:49:59.898196 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:49:59.898248 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:49:59.898416 (+ 168us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:49:59.898510 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:49:59.898575 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0807 08:49:59.898620 (+ 45us) server_negotiation.cc:299] Negotiation successful
0807 08:49:59.898639 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260807 08:49:59.900306 24309 raft_consensus.cc:1261] T 31dd65f1ab22460599fa81e8d522666f P 9d9685c5fddf4b188be861d45570f4e9 [term 1 FOLLOWER]: Refusing update from remote peer 3b8cb0c4f0b3470896fa34ba169b6540: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:49:59.900581 24713 raft_consensus.cc:1261] T 31dd65f1ab22460599fa81e8d522666f P 9c9cbb529f7e41a09d856763bc915e97 [term 1 FOLLOWER]: Refusing update from remote peer 3b8cb0c4f0b3470896fa34ba169b6540: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:49:59.900593 24838 consensus_queue.cc:1035] T 31dd65f1ab22460599fa81e8d522666f P 3b8cb0c4f0b3470896fa34ba169b6540 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d9685c5fddf4b188be861d45570f4e9" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 44725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:49:59.900820 24838 consensus_queue.cc:1035] T 31dd65f1ab22460599fa81e8d522666f P 3b8cb0c4f0b3470896fa34ba169b6540 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c9cbb529f7e41a09d856763bc915e97" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 37811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:49:59.904183 24845 mvcc.cc:205] Tried to move back new op lower bound from 7315835289189638144 to 7315835289075453952. Current Snapshot: MvccSnapshot[applied={T|T < 7315835289189638144}]
I20260807 08:49:59.904546 24848 mvcc.cc:205] Tried to move back new op lower bound from 7315835289189638144 to 7315835289075453952. Current Snapshot: MvccSnapshot[applied={T|T < 7315835289189638144}]
I20260807 08:49:59.912389 24080 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43296:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:49:59.912472 24080 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43296:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:49:59.913170 24080 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 64b4667e647142b79a9ba6b789b99205: Sending DeleteTablet for 3 replicas of tablet 31dd65f1ab22460599fa81e8d522666f
I20260807 08:49:59.913511 24276 tablet_service.cc:1514] Processing DeleteTablet for tablet 31dd65f1ab22460599fa81e8d522666f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:49:59 UTC) from {username='kudu', principal='kudu/127.4.159.190@KRBTEST.COM'} at 127.0.0.1:44164
I20260807 08:49:59.913671 24659 tablet_service.cc:1514] Processing DeleteTablet for tablet 31dd65f1ab22460599fa81e8d522666f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:49:59 UTC) from {username='kudu', principal='kudu/127.4.159.190@KRBTEST.COM'} at 127.0.0.1:37286
I20260807 08:49:59.913971 24463 tablet_service.cc:1514] Processing DeleteTablet for tablet 31dd65f1ab22460599fa81e8d522666f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:49:59 UTC) from {username='kudu', principal='kudu/127.4.159.190@KRBTEST.COM'} at 127.0.0.1:46128
I20260807 08:49:59.914230 24864 tablet_replica.cc:330] T 31dd65f1ab22460599fa81e8d522666f P 3b8cb0c4f0b3470896fa34ba169b6540: stopping tablet replica
I20260807 08:49:59.914245 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 24135
I20260807 08:49:59.916468 24864 raft_consensus.cc:2229] T 31dd65f1ab22460599fa81e8d522666f P 3b8cb0c4f0b3470896fa34ba169b6540 [term 1 LEADER]: Raft consensus shutting down.
I20260807 08:49:59.916676 24864 raft_consensus.cc:2258] T 31dd65f1ab22460599fa81e8d522666f P 3b8cb0c4f0b3470896fa34ba169b6540 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:49:59.925549 24862 tablet_replica.cc:330] T 31dd65f1ab22460599fa81e8d522666f P 9c9cbb529f7e41a09d856763bc915e97: stopping tablet replica
I20260807 08:49:59.925683 24862 raft_consensus.cc:2229] T 31dd65f1ab22460599fa81e8d522666f P 9c9cbb529f7e41a09d856763bc915e97 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:49:59.925793 24862 raft_consensus.cc:2258] T 31dd65f1ab22460599fa81e8d522666f P 9c9cbb529f7e41a09d856763bc915e97 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:49:59.926280 24862 ts_tablet_manager.cc:1905] T 31dd65f1ab22460599fa81e8d522666f P 9c9cbb529f7e41a09d856763bc915e97: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:49:59.926507 24864 ts_tablet_manager.cc:1905] T 31dd65f1ab22460599fa81e8d522666f P 3b8cb0c4f0b3470896fa34ba169b6540: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:49:59.928397 24862 ts_tablet_manager.cc:1918] T 31dd65f1ab22460599fa81e8d522666f P 9c9cbb529f7e41a09d856763bc915e97: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:49:59.928478 24862 log.cc:1196] T 31dd65f1ab22460599fa81e8d522666f P 9c9cbb529f7e41a09d856763bc915e97: Deleting WAL directory at /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ts-2/wal/wals/31dd65f1ab22460599fa81e8d522666f
I20260807 08:49:59.928643 24864 ts_tablet_manager.cc:1918] T 31dd65f1ab22460599fa81e8d522666f P 3b8cb0c4f0b3470896fa34ba169b6540: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:49:59.928718 24864 log.cc:1196] T 31dd65f1ab22460599fa81e8d522666f P 3b8cb0c4f0b3470896fa34ba169b6540: Deleting WAL directory at /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786092453486433-4734-0/minicluster-data/ts-1/wal/wals/31dd65f1ab22460599fa81e8d522666f
I20260807 08:49:59.928754 24862 ts_tablet_manager.cc:1939] T 31dd65f1ab22460599fa81e8d522666f P 9c9cbb529f7e41a09d856763bc915e97: Deleting consensus metadata
I20260807 08:49:59.928979 24864 ts_tablet_manager.cc:1939] T 31dd65f1ab22460599fa81e8d522666f P 3b8cb0c4f0b3470896fa34ba169b6540: Deleting consensus metadata
I20260807 08:49:59.929180 24057 catalog_manager.cc:4841] TS 9c9cbb529f7e41a09d856763bc915e97 (127.4.159.131:37811): tablet 31dd65f1ab22460599fa81e8d522666f (table test-table [id=8be1d19ee257422d8e4331df268a0b77]) successfully deleted
I20260807 08:49:59.929359 24055 catalog_manager.cc:4841] TS 3b8cb0c4f0b3470896fa34ba169b6540 (127.4.159.130:42241): tablet 31dd65f1ab22460599fa81e8d522666f (table test-table [id=8be1d19ee257422d8e4331df268a0b77]) successfully deleted
W20260807 08:49:59.934266 24060 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.159.129:44725 (error 108)
I20260807 08:49:59.935189 24820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:49:59.935073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.129:44725 (local address 127.0.0.1:44180)
0807 08:49:59.935122 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0807 08:49:59.935156 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.129:44725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260807 08:49:59.935318 24060 catalog_manager.cc:4584] TS 9d9685c5fddf4b188be861d45570f4e9 (127.4.159.129:44725): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 31dd65f1ab22460599fa81e8d522666f: Network error: Client connection negotiation failed: client connection to 127.4.159.129:44725: connect: Connection refused (error 111)
I20260807 08:49:59.935494 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 24393
I20260807 08:49:59.942483 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 24594
I20260807 08:49:59.951896 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 24027
I20260807 08:49:59.960845 4734 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260807 08:50:00.304366 4734 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260807 08:50:00.304462 4734 mini_ranger.cc:66] Stopping Ranger...
I20260807 08:50:00.642107 4734 mini_ranger.cc:68] Stopped Ranger
2026-08-07 08:50:00.642 UTC [22577] LOG: received smart shutdown request
2026-08-07 08:50:00.646 UTC [22577] LOG: background worker "logical replication launcher" (PID 22593) exited with exit code 1
2026-08-07 08:50:00.646 UTC [22583] LOG: shutting down
2026-08-07 08:50:00.768 UTC [22577] LOG: database system is shut down
2026-08-07T08:50:00Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (45873 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786092453486433-4734-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:50:00 dist-test-slave-6zbq krb5kdc[24877](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:50:00 dist-test-slave-6zbq krb5kdc[24877](info): set up 2 sockets
Aug 07 08:50:00 dist-test-slave-6zbq krb5kdc[24877](info): commencing operation
krb5kdc: starting...
W20260807 08:50:01.975885 4734 mini_kdc.cc:121] Time spent starting KDC: real 1.100s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786092453486433-4734-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786092453486433-4734-0/krb5kdc/test-user.keytab.
Aug 07 08:50:02 dist-test-slave-6zbq krb5kdc[24877](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092602, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:50:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:50:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.159.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.159.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.159.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab.
Entry for principal kudu/127.4.159.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab.
I20260807 08:50:02.082283 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786092453486433-4734-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786092453486433-4734-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786092453486433-4734-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.159.190:46833
--webserver_interface=127.4.159.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab
--principal=kudu/127.4.159.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:32785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.159.190:46833,127.4.159.189:45283,127.4.159.188:33441
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:50:02.160256 24903 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:50:02.160436 24903 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:50:02.160456 24903 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:50:02.161549 24903 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:50:02.161588 24903 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:50:02.161600 24903 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:50:02.161612 24903 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:50:02.161623 24903 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:50:02.163100 24903 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:32785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786092453486433-4734-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786092453486433-4734-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.159.190:46833,127.4.159.189:45283,127.4.159.188:33441
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab
--principal=kudu/127.4.159.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.159.190:46833
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.159.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24903
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786092453486433-4734-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:50:02.163313 24903 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:50:02.163515 24903 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:50:02.167780 24912 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:50:02.167788 24917 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:50:02.167798 24913 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:50:02.167968 24903 server_base.cc:1034] running on GCE node
I20260807 08:50:02.168197 24903 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:50:02.168435 24903 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:50:02.169572 24903 hybrid_clock.cc:648] HybridClock initialized: now 1786092602169558 us; error 31 us; skew 500 ppm
Aug 07 08:50:02 dist-test-slave-6zbq krb5kdc[24877](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092602, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:50:02.175532 24903 init.cc:377] Logged in from keytab as kudu/127.4.159.190@KRBTEST.COM (short username kudu)
I20260807 08:50:02.175946 24903 webserver.cc:460] Webserver started at http://127.4.159.190:35019/ using document root <none> and password file <none>
I20260807 08:50:02.176128 24903 fs_manager.cc:359] Metadata directory not provided
I20260807 08:50:02.176173 24903 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:50:02.176291 24903 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:50:02.177259 24903 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786092453486433-4734-0/minicluster-data/master-0/data/instance:
uuid: "03754a6e92c74618a5480ac81d067976"
format_stamp: "Formatted at 2026-08-07 08:50:02 on dist-test-slave-6zbq"
server_key: "74bf2222fb2c5a7b9b8cdebea2bb4ceb"
server_key_iv: "b3a45dd808bf02e2e43141d63e93990f"
server_key_version: "encryptionkey@0"
I20260807 08:50:02.177570 24903 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786092453486433-4734-0/minicluster-data/master-0/wal/instance:
uuid: "03754a6e92c74618a5480ac81d067976"
format_stamp: "Formatted at 2026-08-07 08:50:02 on dist-test-slave-6zbq"
server_key: "74bf2222fb2c5a7b9b8cdebea2bb4ceb"
server_key_iv: "b3a45dd808bf02e2e43141d63e93990f"
server_key_version: "encryptionkey@0"
I20260807 08:50:02.178751 24903 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 08:50:02.179356 24926 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:50:02.179522 24903 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 08:50:02.179605 24903 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786092453486433-4734-0/minicluster-data/master-0/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786092453486433-4734-0/minicluster-data/master-0/wal
uuid: "03754a6e92c74618a5480ac81d067976"
format_stamp: "Formatted at 2026-08-07 08:50:02 on dist-test-slave-6zbq"
server_key: "74bf2222fb2c5a7b9b8cdebea2bb4ceb"
server_key_iv: "b3a45dd808bf02e2e43141d63e93990f"
server_key_version: "encryptionkey@0"
I20260807 08:50:02.179662 24903 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786092453486433-4734-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786092453486433-4734-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786092453486433-4734-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:50:02.197605 24903 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:50:02.199187 24903 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:50:02.199311 24903 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:50:02.203003 24903 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.190:46833
I20260807 08:50:02.203032 25002 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.190:46833 every 8 connection(s)
I20260807 08:50:02.203325 24903 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
I20260807 08:50:02.204082 25003 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:50:02.205463 25003 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.159.190" port: 46833 } has no permanent_uuid. Determining permanent_uuid...
I20260807 08:50:02.206152 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 24903
I20260807 08:50:02.206255 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786092453486433-4734-0/minicluster-data/master-0/wal/instance
I20260807 08:50:02.206470 4734 external_mini_cluster.cc:1428] Setting key 5e950808d1067051b1a6f494889166c1
Aug 07 08:50:02 dist-test-slave-6zbq krb5kdc[24877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092602, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.190@KRBTEST.COM for kudu/127.4.159.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.4.159.189@KRBTEST.COM; defaulting to no policy
I20260807 08:50:02.213953 25006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:02.206037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:46833 (local address 127.0.0.1:37316)
0807 08:50:02.206216 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:02.206259 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 08:50:02.206754 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:50:02.206996 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:50:02.206999 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:50:02.207255 (+ 256us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:50:02.207399 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:50:02.207403 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:50:02.208346 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:50:02.208347 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:50:02.208993 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:50:02.208996 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:50:02.209440 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:50:02.209441 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:50:02.209506 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:50:02.209920 (+ 414us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:50:02.209928 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:50:02.211179 (+ 1251us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:50:02.212671 (+ 1492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:50:02.212677 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:50:02.212679 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:50:02.212853 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:50:02.213051 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:50:02.213051 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:50:02.213052 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:50:02.213128 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:50:02.213426 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:50:02.213431 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:50:02.213489 (+ 58us) client_negotiation.cc:769] Sending connection context
0807 08:50:02.213567 (+ 78us) client_negotiation.cc:240] Negotiation successful
0807 08:50:02.213771 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"mutex_wait_us":79,"thread_start_us":80,"threads_started":1}
I20260807 08:50:02.213958 25008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:02.206042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37316 (local address 127.4.159.190:46833)
0807 08:50:02.206208 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0807 08:50:02.206219 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 08:50:02.206396 (+ 177us) server_negotiation.cc:407] Connection header received
0807 08:50:02.206895 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:50:02.206897 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:50:02.206913 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:50:02.206984 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:50:02.207487 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:02.208262 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:02.209067 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:02.209368 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:02.209457 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:50:02.211256 (+ 1799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:50:02.211263 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:50:02.211263 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:50:02.211270 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:50:02.212598 (+ 1328us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:50:02.212921 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:50:02.212925 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:50:02.212925 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:50:02.212995 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:50:02.213181 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:50:02.213181 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:50:02.213182 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:50:02.213365 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:50:02.213433 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 08:50:02.213600 (+ 167us) server_negotiation.cc:299] Negotiation successful
0807 08:50:02.213700 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":78,"threads_started":1}
I20260807 08:50:02.214635 25003 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.159.189" port: 45283 } has no permanent_uuid. Determining permanent_uuid...
I20260807 08:50:02.214915 25006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:02.214817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.189:45283 (local address 127.0.0.1:42954)
0807 08:50:02.214855 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:02.214895 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.189:45283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260807 08:50:02.215021 24929 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.159.189:45283: connect: Connection refused (error 111)
I20260807 08:50:02.215340 25006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:02.215255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.189:45283 (local address 127.0.0.1:42956)
0807 08:50:02.215296 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:02.215319 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.189:45283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260807 08:50:02.215463 25003 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.159.189:45283: Network error: Client connection negotiation failed: client connection to 127.4.159.189:45283: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.159.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.159.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.189.keytab.
Entry for principal kudu/127.4.159.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.189.keytab.
I20260807 08:50:02.228150 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786092453486433-4734-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786092453486433-4734-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786092453486433-4734-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786092453486433-4734-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.159.189:45283
--webserver_interface=127.4.159.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.189.keytab
--principal=kudu/127.4.159.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:32785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.159.190:46833,127.4.159.189:45283,127.4.159.188:33441
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 08:50:02.264675 25003 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.159.189" port: 45283 } attempt: 1
I20260807 08:50:02.265138 25006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:02.265007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.189:45283 (local address 127.0.0.1:42970)
0807 08:50:02.265063 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:02.265106 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.189:45283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260807 08:50:02.265456 25006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:02.265371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.189:45283 (local address 127.0.0.1:42980)
0807 08:50:02.265410 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:02.265436 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.189:45283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260807 08:50:02.265532 25003 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.159.189:45283: Network error: Client connection negotiation failed: client connection to 127.4.159.189:45283: connect: Connection refused (error 111)
W20260807 08:50:02.307560 25012 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:50:02.307786 25012 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:50:02.307807 25012 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:50:02.308933 25012 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:50:02.308974 25012 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:50:02.308987 25012 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:50:02.308998 25012 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:50:02.309010 25012 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:50:02.310565 25012 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:32785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786092453486433-4734-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786092453486433-4734-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.4.159.190:46833,127.4.159.189:45283,127.4.159.188:33441
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.189.keytab
--principal=kudu/127.4.159.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.159.189:45283
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786092453486433-4734-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.4.159.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25012
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786092453486433-4734-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:50:02.310807 25012 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:50:02.311009 25012 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:50:02.315549 25022 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:50:02.315505 25025 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:50:02.315531 25012 server_base.cc:1034] running on GCE node
W20260807 08:50:02.315501 25019 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:50:02.315897 25012 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:50:02.316160 25012 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:50:02.317294 25012 hybrid_clock.cc:648] HybridClock initialized: now 1786092602317286 us; error 27 us; skew 500 ppm
Aug 07 08:50:02 dist-test-slave-6zbq krb5kdc[24877](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092602, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:50:02.319571 25012 init.cc:377] Logged in from keytab as kudu/127.4.159.189@KRBTEST.COM (short username kudu)
I20260807 08:50:02.319937 25012 webserver.cc:460] Webserver started at http://127.4.159.189:41859/ using document root <none> and password file <none>
I20260807 08:50:02.320101 25012 fs_manager.cc:359] Metadata directory not provided
I20260807 08:50:02.320137 25012 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:50:02.320225 25012 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:50:02.321200 25012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786092453486433-4734-0/minicluster-data/master-1/data/instance:
uuid: "82000b497bdf459ebd48b008e0de4ef6"
format_stamp: "Formatted at 2026-08-07 08:50:02 on dist-test-slave-6zbq"
server_key: "856e8e1e4c0d02309cbcd5632e14eb06"
server_key_iv: "7ba4c5e7508ab7bad9570a89818ad442"
server_key_version: "encryptionkey@0"
I20260807 08:50:02.321492 25012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786092453486433-4734-0/minicluster-data/master-1/wal/instance:
uuid: "82000b497bdf459ebd48b008e0de4ef6"
format_stamp: "Formatted at 2026-08-07 08:50:02 on dist-test-slave-6zbq"
server_key: "856e8e1e4c0d02309cbcd5632e14eb06"
server_key_iv: "7ba4c5e7508ab7bad9570a89818ad442"
server_key_version: "encryptionkey@0"
I20260807 08:50:02.322672 25012 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 08:50:02.323300 25038 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:50:02.323446 25012 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 08:50:02.323522 25012 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786092453486433-4734-0/minicluster-data/master-1/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786092453486433-4734-0/minicluster-data/master-1/wal
uuid: "82000b497bdf459ebd48b008e0de4ef6"
format_stamp: "Formatted at 2026-08-07 08:50:02 on dist-test-slave-6zbq"
server_key: "856e8e1e4c0d02309cbcd5632e14eb06"
server_key_iv: "7ba4c5e7508ab7bad9570a89818ad442"
server_key_version: "encryptionkey@0"
I20260807 08:50:02.323585 25012 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786092453486433-4734-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786092453486433-4734-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786092453486433-4734-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:50:02.330679 25012 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:50:02.332244 25012 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:50:02.332368 25012 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:50:02.333654 25003 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.159.189" port: 45283 } attempt: 2
I20260807 08:50:02.333999 25006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:02.333887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.189:45283 (local address 127.0.0.1:42982)
0807 08:50:02.333935 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:02.333964 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.189:45283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260807 08:50:02.334383 25006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:02.334265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.189:45283 (local address 127.0.0.1:42998)
0807 08:50:02.334306 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:02.334358 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.189:45283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260807 08:50:02.334472 25003 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.159.189:45283: Network error: Client connection negotiation failed: client connection to 127.4.159.189:45283: connect: Connection refused (error 111)
I20260807 08:50:02.336185 25012 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.189:45283
I20260807 08:50:02.336223 25122 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.189:45283 every 8 connection(s)
I20260807 08:50:02.336513 25012 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786092453486433-4734-0/minicluster-data/master-1/data/info.pb
I20260807 08:50:02.337293 25123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:50:02.338745 25123 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.159.190" port: 46833 } has no permanent_uuid. Determining permanent_uuid...
I20260807 08:50:02.342195 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 25012
I20260807 08:50:02.342294 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786092453486433-4734-0/minicluster-data/master-1/wal/instance
I20260807 08:50:02.342476 4734 external_mini_cluster.cc:1428] Setting key af44a4346627281ab696ff49043ec12c
Aug 07 08:50:02 dist-test-slave-6zbq krb5kdc[24877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092602, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.189@KRBTEST.COM for kudu/127.4.159.190@KRBTEST.COM
I20260807 08:50:02.347277 25008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:02.339335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37318 (local address 127.4.159.190:46833)
0807 08:50:02.339382 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0807 08:50:02.339384 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:50:02.339573 (+ 189us) server_negotiation.cc:407] Connection header received
0807 08:50:02.340017 (+ 444us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:50:02.340019 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:50:02.340030 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:50:02.340080 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:50:02.340628 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:02.341124 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:02.342092 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:02.342316 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:02.342399 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:50:02.344505 (+ 2106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:50:02.344507 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:50:02.344507 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:50:02.344515 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:50:02.345866 (+ 1351us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:50:02.346258 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:50:02.346259 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:50:02.346260 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:50:02.346322 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:50:02.346617 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:50:02.346618 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:50:02.346618 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:50:02.346769 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:50:02.346841 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 08:50:02.347102 (+ 261us) server_negotiation.cc:299] Negotiation successful
0807 08:50:02.347186 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260807 08:50:02.347358 25125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:02.339325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:46833 (local address 127.0.0.1:37318)
0807 08:50:02.339470 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:02.339503 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0807 08:50:02.339931 (+ 428us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:50:02.340139 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:50:02.340142 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:50:02.340368 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:50:02.340545 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:50:02.340549 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:50:02.341205 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:50:02.341205 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:50:02.342017 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:50:02.342019 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:50:02.342461 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:50:02.342462 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:50:02.342586 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:50:02.342989 (+ 403us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:50:02.343006 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:50:02.344417 (+ 1411us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:50:02.345956 (+ 1539us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:50:02.345962 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:50:02.345964 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:50:02.346185 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:50:02.346433 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:50:02.346434 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:50:02.346435 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:50:02.346546 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:50:02.346857 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:50:02.346863 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:50:02.346979 (+ 116us) client_negotiation.cc:769] Sending connection context
0807 08:50:02.347066 (+ 87us) client_negotiation.cc:240] Negotiation successful
0807 08:50:02.347188 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":69,"threads_started":1}
I20260807 08:50:02.347888 25123 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.159.189" port: 45283 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.4.159.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.159.188@KRBTEST.COM" created.
Aug 07 08:50:02 dist-test-slave-6zbq krb5kdc[24877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092602, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.189@KRBTEST.COM for kudu/127.4.159.189@KRBTEST.COM
I20260807 08:50:02.356599 25125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:02.348097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.189:45283 (local address 127.0.0.1:43014)
0807 08:50:02.348144 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:02.348155 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 08:50:02.348199 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:50:02.348404 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:50:02.348405 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:50:02.348544 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:50:02.348605 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:50:02.348607 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:50:02.349397 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:50:02.349398 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:50:02.350164 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:50:02.350167 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:50:02.350651 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:50:02.350652 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:50:02.350725 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:50:02.351045 (+ 320us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:50:02.351050 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:50:02.354036 (+ 2986us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:50:02.355732 (+ 1696us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:50:02.355732 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:50:02.355733 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:50:02.355884 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:50:02.356082 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:50:02.356082 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:50:02.356083 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:50:02.356123 (+ 40us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:50:02.356403 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:50:02.356404 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:50:02.356434 (+ 30us) client_negotiation.cc:769] Sending connection context
0807 08:50:02.356476 (+ 42us) client_negotiation.cc:240] Negotiation successful
0807 08:50:02.356541 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260807 08:50:02.356712 25127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:02.348155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43014 (local address 127.4.159.189:45283)
0807 08:50:02.348280 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0807 08:50:02.348285 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:50:02.348301 (+ 16us) server_negotiation.cc:407] Connection header received
0807 08:50:02.348318 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:50:02.348320 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:50:02.348330 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:50:02.348386 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:50:02.348669 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:02.349320 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:02.350240 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:02.350583 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:02.350634 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:50:02.354112 (+ 3478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:50:02.354120 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:50:02.354120 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:50:02.354126 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:50:02.355654 (+ 1528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:50:02.355958 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:50:02.355961 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:50:02.355962 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:50:02.356020 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:50:02.356175 (+ 155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:50:02.356176 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:50:02.356176 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:50:02.356346 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:50:02.356427 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 08:50:02.356509 (+ 82us) server_negotiation.cc:299] Negotiation successful
0807 08:50:02.356615 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":18,"server-negotiator.queue_time_us":99,"thread_start_us":65,"threads_started":1}
I20260807 08:50:02.357270 25123 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.159.188" port: 33441 } has no permanent_uuid. Determining permanent_uuid...
I20260807 08:50:02.357553 25125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:02.357447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.188:33441 (local address 127.0.0.1:44860)
0807 08:50:02.357481 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:02.357529 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.188:33441: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
W20260807 08:50:02.357682 25040 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.159.188:33441: connect: Connection refused (error 111)
I20260807 08:50:02.358086 25125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:02.357992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.188:33441 (local address 127.0.0.1:44866)
0807 08:50:02.358035 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:02.358065 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.188:33441: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260807 08:50:02.358188 25123 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.159.188:33441: Network error: Client connection negotiation failed: client connection to 127.4.159.188:33441: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.159.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.188.keytab.
Entry for principal kudu/127.4.159.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.188.keytab.
I20260807 08:50:02.366710 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786092453486433-4734-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786092453486433-4734-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786092453486433-4734-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786092453486433-4734-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.159.188:33441
--webserver_interface=127.4.159.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.188.keytab
--principal=kudu/127.4.159.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:32785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.159.190:46833,127.4.159.189:45283,127.4.159.188:33441
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 08:50:02.407429 25123 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.159.188" port: 33441 } attempt: 1
I20260807 08:50:02.407969 25125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:02.407813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.188:33441 (local address 127.0.0.1:44880)
0807 08:50:02.407882 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:02.407935 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.188:33441: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260807 08:50:02.408452 25125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:02.408307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.188:33441 (local address 127.0.0.1:44882)
0807 08:50:02.408381 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:02.408419 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.188:33441: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260807 08:50:02.408607 25123 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.159.188:33441: Network error: Client connection negotiation failed: client connection to 127.4.159.188:33441: connect: Connection refused (error 111)
I20260807 08:50:02.425601 25003 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.159.189" port: 45283 } attempt: 3
Aug 07 08:50:02 dist-test-slave-6zbq krb5kdc[24877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092602, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.190@KRBTEST.COM for kudu/127.4.159.189@KRBTEST.COM
I20260807 08:50:02.432183 25006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:02.425986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.189:45283 (local address 127.0.0.1:43028)
0807 08:50:02.426035 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:02.426048 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:50:02.426099 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:50:02.426248 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:50:02.426248 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:50:02.426447 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:50:02.426516 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:50:02.426518 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:50:02.427240 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:50:02.427242 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:50:02.427764 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:50:02.427766 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:50:02.428116 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:50:02.428117 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:50:02.428191 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:50:02.428605 (+ 414us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:50:02.428611 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:50:02.429792 (+ 1181us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:50:02.431284 (+ 1492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:50:02.431285 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:50:02.431286 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:50:02.431443 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:50:02.431609 (+ 166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:50:02.431610 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:50:02.431610 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:50:02.431669 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:50:02.431878 (+ 209us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:50:02.431878 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:50:02.431924 (+ 46us) client_negotiation.cc:769] Sending connection context
0807 08:50:02.431985 (+ 61us) client_negotiation.cc:240] Negotiation successful
0807 08:50:02.432083 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260807 08:50:02.432181 25127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:02.426044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43028 (local address 127.4.159.189:45283)
0807 08:50:02.426109 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0807 08:50:02.426111 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:50:02.426122 (+ 11us) server_negotiation.cc:407] Connection header received
0807 08:50:02.426153 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:50:02.426154 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:50:02.426165 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:50:02.426237 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:50:02.426591 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:02.427163 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:02.427846 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:02.428055 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:02.428111 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:50:02.429874 (+ 1763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:50:02.429875 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:50:02.429875 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:50:02.429886 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:50:02.431208 (+ 1322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:50:02.431503 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:50:02.431504 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:50:02.431505 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:50:02.431555 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:50:02.431718 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:50:02.431719 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:50:02.431719 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:50:02.431825 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:50:02.431879 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 08:50:02.432011 (+ 132us) server_negotiation.cc:299] Negotiation successful
0807 08:50:02.432083 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 08:50:02.432544 25003 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.159.188" port: 33441 } has no permanent_uuid. Determining permanent_uuid...
I20260807 08:50:02.432845 25006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:02.432752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.188:33441 (local address 127.0.0.1:44884)
0807 08:50:02.432793 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:02.432819 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.188:33441: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260807 08:50:02.433193 25006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:02.433095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.188:33441 (local address 127.0.0.1:44890)
0807 08:50:02.433156 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:02.433179 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.188:33441: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260807 08:50:02.433282 25003 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.159.188:33441: Network error: Client connection negotiation failed: client connection to 127.4.159.188:33441: connect: Connection refused (error 111)
W20260807 08:50:02.447329 25130 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:50:02.447501 25130 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:50:02.447521 25130 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:50:02.448556 25130 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:50:02.448588 25130 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:50:02.448601 25130 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:50:02.448621 25130 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:50:02.448632 25130 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:50:02.450057 25130 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:32785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786092453486433-4734-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786092453486433-4734-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.4.159.190:46833,127.4.159.189:45283,127.4.159.188:33441
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.188.keytab
--principal=kudu/127.4.159.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.159.188:33441
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786092453486433-4734-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.4.159.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25130
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786092453486433-4734-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:50:02.450239 25130 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:50:02.450438 25130 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:50:02.454748 25138 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:50:02.454739 25137 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:50:02.454739 25142 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:50:02.454891 25130 server_base.cc:1034] running on GCE node
I20260807 08:50:02.455207 25130 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:50:02.455470 25130 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:50:02.456601 25130 hybrid_clock.cc:648] HybridClock initialized: now 1786092602456587 us; error 26 us; skew 500 ppm
Aug 07 08:50:02 dist-test-slave-6zbq krb5kdc[24877](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092602, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:50:02.458829 25130 init.cc:377] Logged in from keytab as kudu/127.4.159.188@KRBTEST.COM (short username kudu)
I20260807 08:50:02.459295 25130 webserver.cc:460] Webserver started at http://127.4.159.188:34743/ using document root <none> and password file <none>
I20260807 08:50:02.459479 25130 fs_manager.cc:359] Metadata directory not provided
I20260807 08:50:02.459525 25130 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:50:02.459647 25130 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:50:02.460654 25130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786092453486433-4734-0/minicluster-data/master-2/data/instance:
uuid: "e7014a60f77745fe85285f528a119334"
format_stamp: "Formatted at 2026-08-07 08:50:02 on dist-test-slave-6zbq"
server_key: "5a5c0510dd103314278f7b4d88798574"
server_key_iv: "2a42808209c706df624f35eaf3d153bc"
server_key_version: "encryptionkey@0"
I20260807 08:50:02.460970 25130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786092453486433-4734-0/minicluster-data/master-2/wal/instance:
uuid: "e7014a60f77745fe85285f528a119334"
format_stamp: "Formatted at 2026-08-07 08:50:02 on dist-test-slave-6zbq"
server_key: "5a5c0510dd103314278f7b4d88798574"
server_key_iv: "2a42808209c706df624f35eaf3d153bc"
server_key_version: "encryptionkey@0"
I20260807 08:50:02.462136 25130 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 08:50:02.462831 25154 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:50:02.462973 25130 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 08:50:02.463053 25130 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786092453486433-4734-0/minicluster-data/master-2/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786092453486433-4734-0/minicluster-data/master-2/wal
uuid: "e7014a60f77745fe85285f528a119334"
format_stamp: "Formatted at 2026-08-07 08:50:02 on dist-test-slave-6zbq"
server_key: "5a5c0510dd103314278f7b4d88798574"
server_key_iv: "2a42808209c706df624f35eaf3d153bc"
server_key_version: "encryptionkey@0"
I20260807 08:50:02.463117 25130 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786092453486433-4734-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786092453486433-4734-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786092453486433-4734-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:50:02.464442 25003 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.159.188" port: 33441 } attempt: 1
I20260807 08:50:02.464776 25006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:02.464667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.188:33441 (local address 127.0.0.1:44898)
0807 08:50:02.464717 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:02.464749 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.188:33441: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260807 08:50:02.465050 25006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:02.464996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.188:33441 (local address 127.0.0.1:44912)
0807 08:50:02.465018 (+ 22us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:02.465037 (+ 19us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.188:33441: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":13}
W20260807 08:50:02.465126 25003 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.159.188:33441: Network error: Client connection negotiation failed: client connection to 127.4.159.188:33441: connect: Connection refused (error 111)
I20260807 08:50:02.476740 25123 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.159.188" port: 33441 } attempt: 2
I20260807 08:50:02.477079 25125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:02.476962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.188:33441 (local address 127.0.0.1:44920)
0807 08:50:02.477013 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:02.477047 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.188:33441: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260807 08:50:02.477360 25125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:02.477297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.188:33441 (local address 127.0.0.1:44936)
0807 08:50:02.477326 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:02.477347 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.188:33441: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260807 08:50:02.477437 25123 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.159.188:33441: Network error: Client connection negotiation failed: client connection to 127.4.159.188:33441: connect: Connection refused (error 111)
I20260807 08:50:02.480098 25130 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:50:02.481964 25130 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:50:02.482084 25130 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:50:02.485675 25130 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.188:33441
I20260807 08:50:02.485704 25227 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.188:33441 every 8 connection(s)
I20260807 08:50:02.485984 25130 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786092453486433-4734-0/minicluster-data/master-2/data/info.pb
I20260807 08:50:02.486759 25232 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:50:02.488097 25232 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.159.190" port: 46833 } has no permanent_uuid. Determining permanent_uuid...
I20260807 08:50:02.490898 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 25130
I20260807 08:50:02.490979 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786092453486433-4734-0/minicluster-data/master-2/wal/instance
I20260807 08:50:02.491145 4734 external_mini_cluster.cc:1428] Setting key 70762f3af73a193e0da55167a253af5e
Aug 07 08:50:02 dist-test-slave-6zbq krb5kdc[24877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092602, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.188@KRBTEST.COM for kudu/127.4.159.190@KRBTEST.COM
Aug 07 08:50:02 dist-test-slave-6zbq krb5kdc[24877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092602, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.159.188@KRBTEST.COM
I20260807 08:50:02.496356 25008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:02.488669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37328 (local address 127.4.159.190:46833)
0807 08:50:02.488720 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0807 08:50:02.488722 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:50:02.488878 (+ 156us) server_negotiation.cc:407] Connection header received
0807 08:50:02.489293 (+ 415us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:50:02.489294 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:50:02.489302 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:50:02.489350 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:50:02.489840 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:02.490314 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:02.491394 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:02.491659 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:02.491720 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:50:02.493661 (+ 1941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:50:02.493662 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:50:02.493663 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:50:02.493670 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:50:02.495132 (+ 1462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:50:02.495472 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:50:02.495473 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:50:02.495474 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:50:02.495530 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:50:02.495778 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:50:02.495778 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:50:02.495779 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:50:02.495877 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:50:02.495925 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0807 08:50:02.496234 (+ 309us) server_negotiation.cc:299] Negotiation successful
0807 08:50:02.496300 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260807 08:50:02.496556 25235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:02.488638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:46833 (local address 127.0.0.1:37328)
0807 08:50:02.488775 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:02.488811 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 08:50:02.489206 (+ 395us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:50:02.489396 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:50:02.489399 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:50:02.489604 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:50:02.489783 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:50:02.489786 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:50:02.490423 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:50:02.490424 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:50:02.491309 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:50:02.491311 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:50:02.491731 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:50:02.491732 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:50:02.491863 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:50:02.492259 (+ 396us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:50:02.492268 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:50:02.493586 (+ 1318us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:50:02.495205 (+ 1619us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:50:02.495210 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:50:02.495211 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:50:02.495408 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:50:02.495596 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:50:02.495596 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:50:02.495598 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:50:02.495721 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:50:02.495935 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:50:02.495940 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:50:02.496077 (+ 137us) client_negotiation.cc:769] Sending connection context
0807 08:50:02.496210 (+ 133us) client_negotiation.cc:240] Negotiation successful
0807 08:50:02.496352 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":68,"threads_started":1}
I20260807 08:50:02.497042 25232 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.159.189" port: 45283 } has no permanent_uuid. Determining permanent_uuid...
I20260807 08:50:02.498996 25237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:02.491445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44950 (local address 127.4.159.188:33441)
0807 08:50:02.491694 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0807 08:50:02.491698 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:50:02.491712 (+ 14us) server_negotiation.cc:407] Connection header received
0807 08:50:02.491728 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:50:02.491729 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:50:02.491744 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:50:02.491810 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:50:02.492351 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:02.493125 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:02.493968 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:02.494300 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:02.494376 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:50:02.496598 (+ 2222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:50:02.496606 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:50:02.496607 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:50:02.496614 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:50:02.497964 (+ 1350us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:50:02.498291 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:50:02.498293 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:50:02.498294 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:50:02.498356 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:50:02.498530 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:50:02.498530 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:50:02.498531 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:50:02.498651 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:50:02.498774 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0807 08:50:02.498875 (+ 101us) server_negotiation.cc:299] Negotiation successful
0807 08:50:02.498931 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":226,"thread_start_us":67,"threads_started":1}
Aug 07 08:50:02 dist-test-slave-6zbq krb5kdc[24877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092602, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.188@KRBTEST.COM for kudu/127.4.159.189@KRBTEST.COM
Aug 07 08:50:02 dist-test-slave-6zbq krb5kdc[24877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092602, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.159.190@KRBTEST.COM
I20260807 08:50:02.503621 25235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:02.497273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.189:45283 (local address 127.0.0.1:43032)
0807 08:50:02.497500 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:02.497508 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0807 08:50:02.497547 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:50:02.497663 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:50:02.497664 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:50:02.497796 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:50:02.497820 (+ 24us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:50:02.497821 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:50:02.498571 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:50:02.498572 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:50:02.499237 (+ 665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:50:02.499239 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:50:02.499582 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:50:02.499583 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:50:02.499879 (+ 296us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:50:02.500227 (+ 348us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:50:02.500231 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:50:02.501316 (+ 1085us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:50:02.502797 (+ 1481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:50:02.502798 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:50:02.502798 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:50:02.502950 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:50:02.503108 (+ 158us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:50:02.503109 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:50:02.503109 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:50:02.503161 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:50:02.503360 (+ 199us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:50:02.503361 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:50:02.503399 (+ 38us) client_negotiation.cc:769] Sending connection context
0807 08:50:02.503459 (+ 60us) client_negotiation.cc:240] Negotiation successful
0807 08:50:02.503536 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216}
I20260807 08:50:02.503630 25127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:02.497325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43032 (local address 127.4.159.189:45283)
0807 08:50:02.497366 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0807 08:50:02.497367 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 08:50:02.497555 (+ 188us) server_negotiation.cc:407] Connection header received
0807 08:50:02.497598 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:50:02.497599 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:50:02.497609 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:50:02.497660 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:50:02.497880 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:02.498517 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:02.499308 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:02.499519 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:02.499582 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:50:02.501384 (+ 1802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:50:02.501386 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:50:02.501386 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:50:02.501392 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:50:02.502729 (+ 1337us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:50:02.503009 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:50:02.503010 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:50:02.503010 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:50:02.503060 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:50:02.503209 (+ 149us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:50:02.503210 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:50:02.503211 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:50:02.503311 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:50:02.503362 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0807 08:50:02.503486 (+ 124us) server_negotiation.cc:299] Negotiation successful
0807 08:50:02.503549 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260807 08:50:02.504177 25232 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.159.188" port: 33441 } has no permanent_uuid. Determining permanent_uuid...
I20260807 08:50:02.506525 25008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:02.499727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37336 (local address 127.4.159.190:46833)
0807 08:50:02.499758 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0807 08:50:02.499760 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:50:02.499767 (+ 7us) server_negotiation.cc:407] Connection header received
0807 08:50:02.499820 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:50:02.499821 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:50:02.499829 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:50:02.499876 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:50:02.500383 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:02.500910 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:02.501586 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:02.501822 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:02.501889 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:50:02.504002 (+ 2113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:50:02.504004 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:50:02.504004 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:50:02.504011 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:50:02.505543 (+ 1532us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:50:02.505816 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:50:02.505817 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:50:02.505818 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:50:02.505877 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:50:02.506040 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:50:02.506041 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:50:02.506042 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:50:02.506145 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:50:02.506308 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0807 08:50:02.506364 (+ 56us) server_negotiation.cc:299] Negotiation successful
0807 08:50:02.506432 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
Aug 07 08:50:02 dist-test-slave-6zbq krb5kdc[24877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092602, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.188@KRBTEST.COM for kudu/127.4.159.188@KRBTEST.COM
I20260807 08:50:02.509989 25235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:02.504381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.188:33441 (local address 127.0.0.1:44962)
0807 08:50:02.504417 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:02.504425 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0807 08:50:02.504454 (+ 29us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:50:02.504572 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:50:02.504572 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:50:02.504667 (+ 95us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:50:02.504691 (+ 24us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:50:02.504692 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:50:02.505475 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:50:02.505476 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:50:02.505910 (+ 434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:50:02.505911 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:50:02.506234 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:50:02.506235 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:50:02.506279 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:50:02.506593 (+ 314us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:50:02.506598 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:50:02.507703 (+ 1105us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:50:02.508995 (+ 1292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:50:02.508996 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:50:02.508996 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:50:02.509135 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:50:02.509460 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:50:02.509461 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:50:02.509461 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:50:02.509511 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:50:02.509706 (+ 195us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:50:02.509706 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:50:02.509740 (+ 34us) client_negotiation.cc:769] Sending connection context
0807 08:50:02.509798 (+ 58us) client_negotiation.cc:240] Negotiation successful
0807 08:50:02.509882 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
Aug 07 08:50:02 dist-test-slave-6zbq krb5kdc[24877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092602, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.159.189@KRBTEST.COM
I20260807 08:50:02.510039 25237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:02.504420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44962 (local address 127.4.159.188:33441)
0807 08:50:02.504467 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0807 08:50:02.504469 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:50:02.504479 (+ 10us) server_negotiation.cc:407] Connection header received
0807 08:50:02.504497 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:50:02.504498 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:50:02.504508 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:50:02.504571 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:50:02.504751 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:02.505400 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:02.505986 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:02.506181 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:02.506231 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:50:02.507777 (+ 1546us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:50:02.507778 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:50:02.507778 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:50:02.507792 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:50:02.508926 (+ 1134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:50:02.509191 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:50:02.509192 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:50:02.509192 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:50:02.509234 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:50:02.509568 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:50:02.509569 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:50:02.509569 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:50:02.509656 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:50:02.509706 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0807 08:50:02.509830 (+ 124us) server_negotiation.cc:299] Negotiation successful
0807 08:50:02.509929 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":39,"server-negotiator.queue_time_us":28}
I20260807 08:50:02.512140 25232 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e7014a60f77745fe85285f528a119334: Bootstrap starting.
I20260807 08:50:02.513077 25232 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e7014a60f77745fe85285f528a119334: Neither blocks nor log segments found. Creating new log.
I20260807 08:50:02.513289 25127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:02.507177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43044 (local address 127.4.159.189:45283)
0807 08:50:02.507218 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0807 08:50:02.507220 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:50:02.507250 (+ 30us) server_negotiation.cc:407] Connection header received
0807 08:50:02.507315 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:50:02.507315 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:50:02.507323 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:50:02.507366 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:50:02.507880 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:02.508300 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:02.508879 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:02.509070 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:02.509121 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:50:02.510967 (+ 1846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:50:02.510968 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:50:02.510968 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:50:02.510975 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:50:02.512312 (+ 1337us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:50:02.512595 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:50:02.512596 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:50:02.512597 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:50:02.512661 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:50:02.512805 (+ 144us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:50:02.512806 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:50:02.512806 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:50:02.512916 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:50:02.513086 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0807 08:50:02.513129 (+ 43us) server_negotiation.cc:299] Negotiation successful
0807 08:50:02.513213 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260807 08:50:02.513456 25232 log.cc:824] T 00000000000000000000000000000000 P e7014a60f77745fe85285f528a119334: Log is configured to *not* fsync() on all Append() calls
I20260807 08:50:02.514276 25232 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e7014a60f77745fe85285f528a119334: No bootstrap required, opened a new log
I20260807 08:50:02.514599 4734 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 08:50:02.516225 25232 raft_consensus.cc:348] T 00000000000000000000000000000000 P e7014a60f77745fe85285f528a119334 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03754a6e92c74618a5480ac81d067976" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 46833 } } peers { permanent_uuid: "82000b497bdf459ebd48b008e0de4ef6" member_type: VOTER last_known_addr { host: "127.4.159.189" port: 45283 } } peers { permanent_uuid: "e7014a60f77745fe85285f528a119334" member_type: VOTER last_known_addr { host: "127.4.159.188" port: 33441 } }
I20260807 08:50:02.516409 25232 raft_consensus.cc:374] T 00000000000000000000000000000000 P e7014a60f77745fe85285f528a119334 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:50:02.516489 25232 raft_consensus.cc:729] T 00000000000000000000000000000000 P e7014a60f77745fe85285f528a119334 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7014a60f77745fe85285f528a119334, State: Initialized, Role: FOLLOWER
I20260807 08:50:02.516671 25232 consensus_queue.cc:260] T 00000000000000000000000000000000 P e7014a60f77745fe85285f528a119334 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03754a6e92c74618a5480ac81d067976" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 46833 } } peers { permanent_uuid: "82000b497bdf459ebd48b008e0de4ef6" member_type: VOTER last_known_addr { host: "127.4.159.189" port: 45283 } } peers { permanent_uuid: "e7014a60f77745fe85285f528a119334" member_type: VOTER last_known_addr { host: "127.4.159.188" port: 33441 } }
I20260807 08:50:02.516959 25249 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7014a60f77745fe85285f528a119334 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03754a6e92c74618a5480ac81d067976" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 46833 } } peers { permanent_uuid: "82000b497bdf459ebd48b008e0de4ef6" member_type: VOTER last_known_addr { host: "127.4.159.189" port: 45283 } } peers { permanent_uuid: "e7014a60f77745fe85285f528a119334" member_type: VOTER last_known_addr { host: "127.4.159.188" port: 33441 } } }
I20260807 08:50:02.517007 25232 sys_catalog.cc:564] T 00000000000000000000000000000000 P e7014a60f77745fe85285f528a119334 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:50:02.517122 25249 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7014a60f77745fe85285f528a119334 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 08:50:02.519104 25130 master_runner.cc:186] Error getting master registration for 127.4.159.190:46833: OK, instance_id { permanent_uuid: "03754a6e92c74618a5480ac81d067976" instance_seqno: 1786092602201933 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 08:50:02.521802 25130 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260807 08:50:02.522272 25267 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e7014a60f77745fe85285f528a119334: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 08:50:02.522399 25267 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 08:50:02.522653 25008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:02.515988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37344 (local address 127.4.159.190:46833)
0807 08:50:02.516023 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0807 08:50:02.516025 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:50:02.516160 (+ 135us) server_negotiation.cc:407] Connection header received
0807 08:50:02.516266 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:50:02.516267 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:50:02.516278 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:50:02.516326 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:50:02.516878 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:02.517452 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:02.518215 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:02.518515 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:02.518565 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:50:02.519964 (+ 1399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:50:02.519965 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:50:02.519965 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:50:02.519971 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:50:02.521385 (+ 1414us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:50:02.521736 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:50:02.521736 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:50:02.521737 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:50:02.521797 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:50:02.521971 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:50:02.521972 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:50:02.521972 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:50:02.522082 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:50:02.522153 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 08:50:02.522498 (+ 345us) server_negotiation.cc:299] Negotiation successful
0807 08:50:02.522570 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260807 08:50:02.523921 25127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:02.516243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43050 (local address 127.4.159.189:45283)
0807 08:50:02.516275 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0807 08:50:02.516277 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:50:02.516350 (+ 73us) server_negotiation.cc:407] Connection header received
0807 08:50:02.516441 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:50:02.516442 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:50:02.516451 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:50:02.516479 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:50:02.517179 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:02.517810 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:02.518817 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:02.519059 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:02.519101 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:50:02.520500 (+ 1399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:50:02.520501 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:50:02.520501 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:50:02.520508 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:50:02.522009 (+ 1501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:50:02.522440 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:50:02.522441 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:50:02.522442 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:50:02.522500 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:50:02.522690 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:50:02.522691 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:50:02.522691 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:50:02.522811 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:50:02.523370 (+ 559us) server_negotiation.cc:1035] Waiting for connection context
0807 08:50:02.523555 (+ 185us) server_negotiation.cc:299] Negotiation successful
0807 08:50:02.523623 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 08:50:02.525694 25237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:02.516158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44966 (local address 127.4.159.188:33441)
0807 08:50:02.516195 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0807 08:50:02.516197 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:50:02.517144 (+ 947us) server_negotiation.cc:407] Connection header received
0807 08:50:02.518473 (+ 1329us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:50:02.518474 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:50:02.518482 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:50:02.518509 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:50:02.519643 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:02.520515 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:02.521998 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:02.522255 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:02.522304 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:50:02.523479 (+ 1175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:50:02.523480 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:50:02.523480 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:50:02.523486 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:50:02.524737 (+ 1251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:50:02.525016 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:50:02.525017 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:50:02.525018 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:50:02.525067 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:50:02.525230 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:50:02.525231 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:50:02.525231 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:50:02.525351 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:50:02.525425 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 08:50:02.525553 (+ 128us) server_negotiation.cc:299] Negotiation successful
0807 08:50:02.525616 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 08:50:02.540261 25003 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.159.188" port: 33441 } attempt: 2
Aug 07 08:50:02 dist-test-slave-6zbq krb5kdc[24877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092602, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.190@KRBTEST.COM for kudu/127.4.159.188@KRBTEST.COM
I20260807 08:50:02.545840 25006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:02.540445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.188:33441 (local address 127.0.0.1:44974)
0807 08:50:02.540482 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:02.540490 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0807 08:50:02.540534 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:50:02.540756 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:50:02.540757 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:50:02.540880 (+ 123us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:50:02.540911 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:50:02.540913 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:50:02.541458 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:50:02.541459 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:50:02.541913 (+ 454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:50:02.541915 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:50:02.542286 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:50:02.542287 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:50:02.542378 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:50:02.542673 (+ 295us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:50:02.542677 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:50:02.543776 (+ 1099us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:50:02.545020 (+ 1244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:50:02.545021 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:50:02.545022 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:50:02.545172 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:50:02.545326 (+ 154us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:50:02.545326 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:50:02.545327 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:50:02.545380 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:50:02.545569 (+ 189us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:50:02.545570 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:50:02.545609 (+ 39us) client_negotiation.cc:769] Sending connection context
0807 08:50:02.545667 (+ 58us) client_negotiation.cc:240] Negotiation successful
0807 08:50:02.545749 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23}
I20260807 08:50:02.545848 25237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:02.540507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44974 (local address 127.4.159.188:33441)
0807 08:50:02.540663 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0807 08:50:02.540664 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 08:50:02.540671 (+ 7us) server_negotiation.cc:407] Connection header received
0807 08:50:02.540688 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:50:02.540688 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:50:02.540695 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:50:02.540740 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:50:02.540973 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:02.541396 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:02.541977 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:02.542173 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:02.542227 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:50:02.543850 (+ 1623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:50:02.543851 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:50:02.543851 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:50:02.543857 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:50:02.544950 (+ 1093us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:50:02.545228 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:50:02.545229 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:50:02.545229 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:50:02.545273 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:50:02.545429 (+ 156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:50:02.545429 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:50:02.545430 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:50:02.545517 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:50:02.545567 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0807 08:50:02.545708 (+ 141us) server_negotiation.cc:299] Negotiation successful
0807 08:50:02.545766 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136}
I20260807 08:50:02.547434 25003 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03754a6e92c74618a5480ac81d067976: Bootstrap starting.
I20260807 08:50:02.548137 25003 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 03754a6e92c74618a5480ac81d067976: Neither blocks nor log segments found. Creating new log.
I20260807 08:50:02.548436 25003 log.cc:824] T 00000000000000000000000000000000 P 03754a6e92c74618a5480ac81d067976: Log is configured to *not* fsync() on all Append() calls
I20260807 08:50:02.549244 25003 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03754a6e92c74618a5480ac81d067976: No bootstrap required, opened a new log
I20260807 08:50:02.551205 25003 raft_consensus.cc:348] T 00000000000000000000000000000000 P 03754a6e92c74618a5480ac81d067976 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03754a6e92c74618a5480ac81d067976" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 46833 } } peers { permanent_uuid: "82000b497bdf459ebd48b008e0de4ef6" member_type: VOTER last_known_addr { host: "127.4.159.189" port: 45283 } } peers { permanent_uuid: "e7014a60f77745fe85285f528a119334" member_type: VOTER last_known_addr { host: "127.4.159.188" port: 33441 } }
I20260807 08:50:02.551342 25003 raft_consensus.cc:374] T 00000000000000000000000000000000 P 03754a6e92c74618a5480ac81d067976 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:50:02.551380 25003 raft_consensus.cc:729] T 00000000000000000000000000000000 P 03754a6e92c74618a5480ac81d067976 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03754a6e92c74618a5480ac81d067976, State: Initialized, Role: FOLLOWER
I20260807 08:50:02.551491 25003 consensus_queue.cc:260] T 00000000000000000000000000000000 P 03754a6e92c74618a5480ac81d067976 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03754a6e92c74618a5480ac81d067976" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 46833 } } peers { permanent_uuid: "82000b497bdf459ebd48b008e0de4ef6" member_type: VOTER last_known_addr { host: "127.4.159.189" port: 45283 } } peers { permanent_uuid: "e7014a60f77745fe85285f528a119334" member_type: VOTER last_known_addr { host: "127.4.159.188" port: 33441 } }
I20260807 08:50:02.551741 25271 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03754a6e92c74618a5480ac81d067976 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03754a6e92c74618a5480ac81d067976" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 46833 } } peers { permanent_uuid: "82000b497bdf459ebd48b008e0de4ef6" member_type: VOTER last_known_addr { host: "127.4.159.189" port: 45283 } } peers { permanent_uuid: "e7014a60f77745fe85285f528a119334" member_type: VOTER last_known_addr { host: "127.4.159.188" port: 33441 } } }
I20260807 08:50:02.551785 25003 sys_catalog.cc:564] T 00000000000000000000000000000000 P 03754a6e92c74618a5480ac81d067976 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:50:02.551839 25271 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03754a6e92c74618a5480ac81d067976 [sys.catalog]: This master's current role is: FOLLOWER
W20260807 08:50:02.553541 25287 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 03754a6e92c74618a5480ac81d067976: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 08:50:02.553634 25287 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 08:50:02.554174 24903 master_runner.cc:186] Error getting master registration for 127.4.159.189:45283: OK, instance_id { permanent_uuid: "82000b497bdf459ebd48b008e0de4ef6" instance_seqno: 1786092602335193 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 08:50:02.556499 24903 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 08:50:02.568567 25123 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.159.188" port: 33441 } attempt: 3
Aug 07 08:50:02 dist-test-slave-6zbq krb5kdc[24877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092602, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.189@KRBTEST.COM for kudu/127.4.159.188@KRBTEST.COM
I20260807 08:50:02.574029 25237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:02.568837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44990 (local address 127.4.159.188:33441)
0807 08:50:02.568869 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0807 08:50:02.568871 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:50:02.568878 (+ 7us) server_negotiation.cc:407] Connection header received
0807 08:50:02.568899 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:50:02.568899 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:50:02.568907 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:50:02.568951 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:50:02.569186 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:02.569606 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:02.570194 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:02.570396 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:02.570444 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:50:02.571952 (+ 1508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:50:02.571953 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:50:02.571953 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:50:02.571960 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:50:02.573080 (+ 1120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:50:02.573365 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:50:02.573366 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:50:02.573366 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:50:02.573411 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:50:02.573601 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:50:02.573602 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:50:02.573602 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:50:02.573694 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:50:02.573747 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0807 08:50:02.573886 (+ 139us) server_negotiation.cc:299] Negotiation successful
0807 08:50:02.573944 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 08:50:02.574033 25125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:02.568768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.188:33441 (local address 127.0.0.1:44990)
0807 08:50:02.568813 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:02.568824 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 08:50:02.568859 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:50:02.568961 (+ 102us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:50:02.568962 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:50:02.569090 (+ 128us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:50:02.569127 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:50:02.569128 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:50:02.569666 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:50:02.569667 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:50:02.570131 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:50:02.570132 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:50:02.570459 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:50:02.570460 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:50:02.570509 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:50:02.570858 (+ 349us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:50:02.570864 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:50:02.571880 (+ 1016us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:50:02.573151 (+ 1271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:50:02.573152 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:50:02.573153 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:50:02.573307 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:50:02.573462 (+ 155us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:50:02.573462 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:50:02.573463 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:50:02.573545 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:50:02.573748 (+ 203us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:50:02.573748 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:50:02.573797 (+ 49us) client_negotiation.cc:769] Sending connection context
0807 08:50:02.573856 (+ 59us) client_negotiation.cc:240] Negotiation successful
0807 08:50:02.573942 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260807 08:50:02.575464 25123 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82000b497bdf459ebd48b008e0de4ef6: Bootstrap starting.
I20260807 08:50:02.576079 25123 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 82000b497bdf459ebd48b008e0de4ef6: Neither blocks nor log segments found. Creating new log.
I20260807 08:50:02.576334 25123 log.cc:824] T 00000000000000000000000000000000 P 82000b497bdf459ebd48b008e0de4ef6: Log is configured to *not* fsync() on all Append() calls
I20260807 08:50:02.577064 25123 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82000b497bdf459ebd48b008e0de4ef6: No bootstrap required, opened a new log
I20260807 08:50:02.578425 25123 raft_consensus.cc:348] T 00000000000000000000000000000000 P 82000b497bdf459ebd48b008e0de4ef6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03754a6e92c74618a5480ac81d067976" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 46833 } } peers { permanent_uuid: "82000b497bdf459ebd48b008e0de4ef6" member_type: VOTER last_known_addr { host: "127.4.159.189" port: 45283 } } peers { permanent_uuid: "e7014a60f77745fe85285f528a119334" member_type: VOTER last_known_addr { host: "127.4.159.188" port: 33441 } }
I20260807 08:50:02.578526 25123 raft_consensus.cc:374] T 00000000000000000000000000000000 P 82000b497bdf459ebd48b008e0de4ef6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:50:02.578557 25123 raft_consensus.cc:729] T 00000000000000000000000000000000 P 82000b497bdf459ebd48b008e0de4ef6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82000b497bdf459ebd48b008e0de4ef6, State: Initialized, Role: FOLLOWER
I20260807 08:50:02.578696 25123 consensus_queue.cc:260] T 00000000000000000000000000000000 P 82000b497bdf459ebd48b008e0de4ef6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03754a6e92c74618a5480ac81d067976" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 46833 } } peers { permanent_uuid: "82000b497bdf459ebd48b008e0de4ef6" member_type: VOTER last_known_addr { host: "127.4.159.189" port: 45283 } } peers { permanent_uuid: "e7014a60f77745fe85285f528a119334" member_type: VOTER last_known_addr { host: "127.4.159.188" port: 33441 } }
I20260807 08:50:02.578934 25290 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82000b497bdf459ebd48b008e0de4ef6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03754a6e92c74618a5480ac81d067976" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 46833 } } peers { permanent_uuid: "82000b497bdf459ebd48b008e0de4ef6" member_type: VOTER last_known_addr { host: "127.4.159.189" port: 45283 } } peers { permanent_uuid: "e7014a60f77745fe85285f528a119334" member_type: VOTER last_known_addr { host: "127.4.159.188" port: 33441 } } }
I20260807 08:50:02.578987 25123 sys_catalog.cc:564] T 00000000000000000000000000000000 P 82000b497bdf459ebd48b008e0de4ef6 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:50:02.579062 25290 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82000b497bdf459ebd48b008e0de4ef6 [sys.catalog]: This master's current role is: FOLLOWER
W20260807 08:50:02.580660 25306 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 82000b497bdf459ebd48b008e0de4ef6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 08:50:02.580756 25306 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 08:50:02.582700 25012 master_runner.cc:263] No leader master found from master 82000b497bdf459ebd48b008e0de4ef6
I20260807 08:50:02.582747 25012 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260807 08:50:02.582858 25049 tablet.cc:2369] T 00000000000000000000000000000000 P 82000b497bdf459ebd48b008e0de4ef6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 08:50:02.699980 24941 tablet.cc:2369] T 00000000000000000000000000000000 P 03754a6e92c74618a5480ac81d067976: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 08:50:02.732614 25161 tablet.cc:2369] T 00000000000000000000000000000000 P e7014a60f77745fe85285f528a119334: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 08:50:02.785085 25249 raft_consensus.cc:482] T 00000000000000000000000000000000 P e7014a60f77745fe85285f528a119334 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:50:02.785188 25249 raft_consensus.cc:504] T 00000000000000000000000000000000 P e7014a60f77745fe85285f528a119334 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03754a6e92c74618a5480ac81d067976" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 46833 } } peers { permanent_uuid: "82000b497bdf459ebd48b008e0de4ef6" member_type: VOTER last_known_addr { host: "127.4.159.189" port: 45283 } } peers { permanent_uuid: "e7014a60f77745fe85285f528a119334" member_type: VOTER last_known_addr { host: "127.4.159.188" port: 33441 } }
I20260807 08:50:02.785374 25249 leader_election.cc:288] T 00000000000000000000000000000000 P e7014a60f77745fe85285f528a119334 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 03754a6e92c74618a5480ac81d067976 (127.4.159.190:46833), 82000b497bdf459ebd48b008e0de4ef6 (127.4.159.189:45283)
I20260807 08:50:02.785595 24965 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e7014a60f77745fe85285f528a119334" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03754a6e92c74618a5480ac81d067976" is_pre_election: true
I20260807 08:50:02.785656 25085 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e7014a60f77745fe85285f528a119334" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "82000b497bdf459ebd48b008e0de4ef6" is_pre_election: true
I20260807 08:50:02.785723 24965 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 03754a6e92c74618a5480ac81d067976 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7014a60f77745fe85285f528a119334 in term 0.
I20260807 08:50:02.785807 25085 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 82000b497bdf459ebd48b008e0de4ef6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7014a60f77745fe85285f528a119334 in term 0.
I20260807 08:50:02.785975 25156 leader_election.cc:302] T 00000000000000000000000000000000 P e7014a60f77745fe85285f528a119334 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 82000b497bdf459ebd48b008e0de4ef6, e7014a60f77745fe85285f528a119334; no voters:
I20260807 08:50:02.786105 25249 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e7014a60f77745fe85285f528a119334 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:50:02.786165 25249 raft_consensus.cc:482] T 00000000000000000000000000000000 P e7014a60f77745fe85285f528a119334 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:50:02.786180 25249 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e7014a60f77745fe85285f528a119334 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:50:02.786866 25249 raft_consensus.cc:504] T 00000000000000000000000000000000 P e7014a60f77745fe85285f528a119334 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03754a6e92c74618a5480ac81d067976" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 46833 } } peers { permanent_uuid: "82000b497bdf459ebd48b008e0de4ef6" member_type: VOTER last_known_addr { host: "127.4.159.189" port: 45283 } } peers { permanent_uuid: "e7014a60f77745fe85285f528a119334" member_type: VOTER last_known_addr { host: "127.4.159.188" port: 33441 } }
I20260807 08:50:02.786972 25249 leader_election.cc:288] T 00000000000000000000000000000000 P e7014a60f77745fe85285f528a119334 [CANDIDATE]: Term 1 election: Requested vote from peers 03754a6e92c74618a5480ac81d067976 (127.4.159.190:46833), 82000b497bdf459ebd48b008e0de4ef6 (127.4.159.189:45283)
I20260807 08:50:02.787122 24965 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e7014a60f77745fe85285f528a119334" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03754a6e92c74618a5480ac81d067976"
I20260807 08:50:02.787161 25085 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e7014a60f77745fe85285f528a119334" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "82000b497bdf459ebd48b008e0de4ef6"
I20260807 08:50:02.787223 24965 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 03754a6e92c74618a5480ac81d067976 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:50:02.787223 25085 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 82000b497bdf459ebd48b008e0de4ef6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:50:02.788085 25085 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 82000b497bdf459ebd48b008e0de4ef6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7014a60f77745fe85285f528a119334 in term 1.
I20260807 08:50:02.788125 24965 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 03754a6e92c74618a5480ac81d067976 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7014a60f77745fe85285f528a119334 in term 1.
I20260807 08:50:02.788254 25156 leader_election.cc:302] T 00000000000000000000000000000000 P e7014a60f77745fe85285f528a119334 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 82000b497bdf459ebd48b008e0de4ef6, e7014a60f77745fe85285f528a119334; no voters:
I20260807 08:50:02.788354 25249 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e7014a60f77745fe85285f528a119334 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:50:02.788525 25249 raft_consensus.cc:686] T 00000000000000000000000000000000 P e7014a60f77745fe85285f528a119334 [term 1 LEADER]: Becoming Leader. State: Replica: e7014a60f77745fe85285f528a119334, State: Running, Role: LEADER
I20260807 08:50:02.788618 25249 consensus_queue.cc:237] T 00000000000000000000000000000000 P e7014a60f77745fe85285f528a119334 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03754a6e92c74618a5480ac81d067976" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 46833 } } peers { permanent_uuid: "82000b497bdf459ebd48b008e0de4ef6" member_type: VOTER last_known_addr { host: "127.4.159.189" port: 45283 } } peers { permanent_uuid: "e7014a60f77745fe85285f528a119334" member_type: VOTER last_known_addr { host: "127.4.159.188" port: 33441 } }
I20260807 08:50:02.788998 25308 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7014a60f77745fe85285f528a119334 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e7014a60f77745fe85285f528a119334. Latest consensus state: current_term: 1 leader_uuid: "e7014a60f77745fe85285f528a119334" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03754a6e92c74618a5480ac81d067976" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 46833 } } peers { permanent_uuid: "82000b497bdf459ebd48b008e0de4ef6" member_type: VOTER last_known_addr { host: "127.4.159.189" port: 45283 } } peers { permanent_uuid: "e7014a60f77745fe85285f528a119334" member_type: VOTER last_known_addr { host: "127.4.159.188" port: 33441 } } }
I20260807 08:50:02.789086 25308 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7014a60f77745fe85285f528a119334 [sys.catalog]: This master's current role is: LEADER
I20260807 08:50:02.789269 25310 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:50:02.789438 25310 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:50:02.790666 25085 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 82000b497bdf459ebd48b008e0de4ef6 [term 1 FOLLOWER]: Refusing update from remote peer e7014a60f77745fe85285f528a119334: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:50:02.790668 24965 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 03754a6e92c74618a5480ac81d067976 [term 1 FOLLOWER]: Refusing update from remote peer e7014a60f77745fe85285f528a119334: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:50:02.790921 25308 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e7014a60f77745fe85285f528a119334 [LEADER]: Connected to new peer: Peer: permanent_uuid: "82000b497bdf459ebd48b008e0de4ef6" member_type: VOTER last_known_addr { host: "127.4.159.189" port: 45283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:50:02.791040 25249 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e7014a60f77745fe85285f528a119334 [LEADER]: Connected to new peer: Peer: permanent_uuid: "03754a6e92c74618a5480ac81d067976" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 46833 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:50:02.792109 25290 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82000b497bdf459ebd48b008e0de4ef6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e7014a60f77745fe85285f528a119334. Latest consensus state: current_term: 1 leader_uuid: "e7014a60f77745fe85285f528a119334" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03754a6e92c74618a5480ac81d067976" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 46833 } } peers { permanent_uuid: "82000b497bdf459ebd48b008e0de4ef6" member_type: VOTER last_known_addr { host: "127.4.159.189" port: 45283 } } peers { permanent_uuid: "e7014a60f77745fe85285f528a119334" member_type: VOTER last_known_addr { host: "127.4.159.188" port: 33441 } } }
I20260807 08:50:02.792124 25271 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03754a6e92c74618a5480ac81d067976 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e7014a60f77745fe85285f528a119334. Latest consensus state: current_term: 1 leader_uuid: "e7014a60f77745fe85285f528a119334" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03754a6e92c74618a5480ac81d067976" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 46833 } } peers { permanent_uuid: "82000b497bdf459ebd48b008e0de4ef6" member_type: VOTER last_known_addr { host: "127.4.159.189" port: 45283 } } peers { permanent_uuid: "e7014a60f77745fe85285f528a119334" member_type: VOTER last_known_addr { host: "127.4.159.188" port: 33441 } } }
I20260807 08:50:02.792189 25290 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82000b497bdf459ebd48b008e0de4ef6 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 08:50:02.792197 25271 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03754a6e92c74618a5480ac81d067976 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 08:50:02.793004 25249 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7014a60f77745fe85285f528a119334 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e7014a60f77745fe85285f528a119334" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03754a6e92c74618a5480ac81d067976" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 46833 } } peers { permanent_uuid: "82000b497bdf459ebd48b008e0de4ef6" member_type: VOTER last_known_addr { host: "127.4.159.189" port: 45283 } } peers { permanent_uuid: "e7014a60f77745fe85285f528a119334" member_type: VOTER last_known_addr { host: "127.4.159.188" port: 33441 } } }
I20260807 08:50:02.793100 25249 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7014a60f77745fe85285f528a119334 [sys.catalog]: This master's current role is: LEADER
I20260807 08:50:02.793012 25308 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7014a60f77745fe85285f528a119334 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e7014a60f77745fe85285f528a119334" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03754a6e92c74618a5480ac81d067976" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 46833 } } peers { permanent_uuid: "82000b497bdf459ebd48b008e0de4ef6" member_type: VOTER last_known_addr { host: "127.4.159.189" port: 45283 } } peers { permanent_uuid: "e7014a60f77745fe85285f528a119334" member_type: VOTER last_known_addr { host: "127.4.159.188" port: 33441 } } }
I20260807 08:50:02.793146 25308 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7014a60f77745fe85285f528a119334 [sys.catalog]: This master's current role is: LEADER
I20260807 08:50:02.793149 25311 mvcc.cc:205] Tried to move back new op lower bound from 7315835301028417536 to 7315835301022650368. Current Snapshot: MvccSnapshot[applied={T|T < 7315835301028417536}]
I20260807 08:50:02.793175 25271 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03754a6e92c74618a5480ac81d067976 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e7014a60f77745fe85285f528a119334" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03754a6e92c74618a5480ac81d067976" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 46833 } } peers { permanent_uuid: "82000b497bdf459ebd48b008e0de4ef6" member_type: VOTER last_known_addr { host: "127.4.159.189" port: 45283 } } peers { permanent_uuid: "e7014a60f77745fe85285f528a119334" member_type: VOTER last_known_addr { host: "127.4.159.188" port: 33441 } } }
I20260807 08:50:02.793195 25290 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82000b497bdf459ebd48b008e0de4ef6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e7014a60f77745fe85285f528a119334" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03754a6e92c74618a5480ac81d067976" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 46833 } } peers { permanent_uuid: "82000b497bdf459ebd48b008e0de4ef6" member_type: VOTER last_known_addr { host: "127.4.159.189" port: 45283 } } peers { permanent_uuid: "e7014a60f77745fe85285f528a119334" member_type: VOTER last_known_addr { host: "127.4.159.188" port: 33441 } } }
I20260807 08:50:02.793251 25271 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03754a6e92c74618a5480ac81d067976 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 08:50:02.793252 25290 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82000b497bdf459ebd48b008e0de4ef6 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 08:50:02.793426 25312 mvcc.cc:205] Tried to move back new op lower bound from 7315835301028417536 to 7315835301022650368. Current Snapshot: MvccSnapshot[applied={T|T < 7315835301028417536}]
I20260807 08:50:02.794242 25310 catalog_manager.cc:1347] Generated new cluster ID: ddc07b4d93da463cb1153692c20c8093
I20260807 08:50:02.794288 25310 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:50:02.802292 25310 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:50:02.802888 25310 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:50:02.809274 25310 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e7014a60f77745fe85285f528a119334: Generated new TSK 0
I20260807 08:50:02.809432 25310 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:50:03.554081 25287 catalog_manager.cc:1259] Loaded cluster ID: ddc07b4d93da463cb1153692c20c8093
I20260807 08:50:03.554147 25287 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 03754a6e92c74618a5480ac81d067976: loading cluster ID for follower catalog manager: success
I20260807 08:50:03.554957 25287 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 03754a6e92c74618a5480ac81d067976: acquiring CA information for follower catalog manager: success
I20260807 08:50:03.555251 25287 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 03754a6e92c74618a5480ac81d067976: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 08:50:03.581290 25306 catalog_manager.cc:1259] Loaded cluster ID: ddc07b4d93da463cb1153692c20c8093
I20260807 08:50:03.581391 25306 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 82000b497bdf459ebd48b008e0de4ef6: loading cluster ID for follower catalog manager: success
I20260807 08:50:03.582273 25306 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 82000b497bdf459ebd48b008e0de4ef6: acquiring CA information for follower catalog manager: success
I20260807 08:50:03.582620 25306 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 82000b497bdf459ebd48b008e0de4ef6: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 08:50:03.864835 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 24903
I20260807 08:50:03.875298 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 25012
I20260807 08:50:03.887585 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 25130
2026-08-07T08:50:03Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3036 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786092453486433-4734-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:50:03 dist-test-slave-6zbq krb5kdc[25335](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:50:03 dist-test-slave-6zbq krb5kdc[25335](info): set up 2 sockets
Aug 07 08:50:03 dist-test-slave-6zbq krb5kdc[25335](info): commencing operation
krb5kdc: starting...
W20260807 08:50:05.017738 4734 mini_kdc.cc:121] Time spent starting KDC: real 1.106s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786092453486433-4734-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786092453486433-4734-0/krb5kdc/test-user.keytab.
Aug 07 08:50:05 dist-test-slave-6zbq krb5kdc[25335](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092605, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:50:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:50:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.159.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.159.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.159.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab.
Entry for principal kudu/127.4.159.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab.
I20260807 08:50:05.124964 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786092453486433-4734-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786092453486433-4734-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786092453486433-4734-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.159.190:43597
--webserver_interface=127.4.159.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab
--principal=kudu/127.4.159.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:34113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.159.190:43597
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:50:05.202682 25351 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:50:05.202872 25351 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:50:05.202901 25351 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:50:05.203987 25351 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:50:05.204025 25351 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:50:05.204046 25351 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:50:05.204061 25351 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:50:05.204077 25351 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:50:05.205492 25351 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:34113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786092453486433-4734-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786092453486433-4734-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.159.190:43597
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab
--principal=kudu/127.4.159.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.159.190:43597
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.159.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25351
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786092453486433-4734-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:50:05.205718 25351 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:50:05.205912 25351 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:50:05.210386 25360 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:50:05.210359 25366 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:50:05.210440 25351 server_base.cc:1034] running on GCE node
W20260807 08:50:05.210367 25361 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:50:05.210846 25351 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:50:05.211126 25351 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:50:05.212260 25351 hybrid_clock.cc:648] HybridClock initialized: now 1786092605212247 us; error 30 us; skew 500 ppm
Aug 07 08:50:05 dist-test-slave-6zbq krb5kdc[25335](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092605, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:50:05.214571 25351 init.cc:377] Logged in from keytab as kudu/127.4.159.190@KRBTEST.COM (short username kudu)
I20260807 08:50:05.214941 25351 webserver.cc:460] Webserver started at http://127.4.159.190:40901/ using document root <none> and password file <none>
I20260807 08:50:05.215096 25351 fs_manager.cc:359] Metadata directory not provided
I20260807 08:50:05.215133 25351 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:50:05.215219 25351 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:50:05.216131 25351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786092453486433-4734-0/minicluster-data/master-0/data/instance:
uuid: "b1f31aafad064cc997b46dfd9b805d96"
format_stamp: "Formatted at 2026-08-07 08:50:05 on dist-test-slave-6zbq"
server_key: "68ecbfe9c93a3c54f2825e1ed17050b1"
server_key_iv: "97ac35f646b3ad73d43fb3222da88b76"
server_key_version: "encryptionkey@0"
I20260807 08:50:05.216400 25351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786092453486433-4734-0/minicluster-data/master-0/wal/instance:
uuid: "b1f31aafad064cc997b46dfd9b805d96"
format_stamp: "Formatted at 2026-08-07 08:50:05 on dist-test-slave-6zbq"
server_key: "68ecbfe9c93a3c54f2825e1ed17050b1"
server_key_iv: "97ac35f646b3ad73d43fb3222da88b76"
server_key_version: "encryptionkey@0"
I20260807 08:50:05.217491 25351 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 08:50:05.218096 25379 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:50:05.218242 25351 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 08:50:05.218322 25351 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786092453486433-4734-0/minicluster-data/master-0/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786092453486433-4734-0/minicluster-data/master-0/wal
uuid: "b1f31aafad064cc997b46dfd9b805d96"
format_stamp: "Formatted at 2026-08-07 08:50:05 on dist-test-slave-6zbq"
server_key: "68ecbfe9c93a3c54f2825e1ed17050b1"
server_key_iv: "97ac35f646b3ad73d43fb3222da88b76"
server_key_version: "encryptionkey@0"
I20260807 08:50:05.218411 25351 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786092453486433-4734-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786092453486433-4734-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786092453486433-4734-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:50:05.233084 25351 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:50:05.235024 25351 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:50:05.235145 25351 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:50:05.238744 25351 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.190:43597
I20260807 08:50:05.238785 25451 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.190:43597 every 8 connection(s)
I20260807 08:50:05.239073 25351 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
I20260807 08:50:05.239849 25454 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:50:05.241916 25454 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b1f31aafad064cc997b46dfd9b805d96: Bootstrap starting.
I20260807 08:50:05.242449 25454 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b1f31aafad064cc997b46dfd9b805d96: Neither blocks nor log segments found. Creating new log.
I20260807 08:50:05.242686 25454 log.cc:824] T 00000000000000000000000000000000 P b1f31aafad064cc997b46dfd9b805d96: Log is configured to *not* fsync() on all Append() calls
I20260807 08:50:05.243326 25454 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b1f31aafad064cc997b46dfd9b805d96: No bootstrap required, opened a new log
I20260807 08:50:05.244835 25454 raft_consensus.cc:348] T 00000000000000000000000000000000 P b1f31aafad064cc997b46dfd9b805d96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1f31aafad064cc997b46dfd9b805d96" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 43597 } }
I20260807 08:50:05.244961 25454 raft_consensus.cc:374] T 00000000000000000000000000000000 P b1f31aafad064cc997b46dfd9b805d96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:50:05.244988 25454 raft_consensus.cc:729] T 00000000000000000000000000000000 P b1f31aafad064cc997b46dfd9b805d96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f31aafad064cc997b46dfd9b805d96, State: Initialized, Role: FOLLOWER
I20260807 08:50:05.245095 25454 consensus_queue.cc:260] T 00000000000000000000000000000000 P b1f31aafad064cc997b46dfd9b805d96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1f31aafad064cc997b46dfd9b805d96" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 43597 } }
I20260807 08:50:05.245170 25454 raft_consensus.cc:388] T 00000000000000000000000000000000 P b1f31aafad064cc997b46dfd9b805d96 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:50:05.245201 25454 raft_consensus.cc:482] T 00000000000000000000000000000000 P b1f31aafad064cc997b46dfd9b805d96 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:50:05.245234 25454 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b1f31aafad064cc997b46dfd9b805d96 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:50:05.245680 25454 raft_consensus.cc:504] T 00000000000000000000000000000000 P b1f31aafad064cc997b46dfd9b805d96 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1f31aafad064cc997b46dfd9b805d96" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 43597 } }
I20260807 08:50:05.245771 25454 leader_election.cc:302] T 00000000000000000000000000000000 P b1f31aafad064cc997b46dfd9b805d96 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b1f31aafad064cc997b46dfd9b805d96; no voters:
I20260807 08:50:05.245884 25454 leader_election.cc:288] T 00000000000000000000000000000000 P b1f31aafad064cc997b46dfd9b805d96 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:50:05.245923 25457 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b1f31aafad064cc997b46dfd9b805d96 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:50:05.246084 25454 sys_catalog.cc:564] T 00000000000000000000000000000000 P b1f31aafad064cc997b46dfd9b805d96 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:50:05.246083 25457 raft_consensus.cc:686] T 00000000000000000000000000000000 P b1f31aafad064cc997b46dfd9b805d96 [term 1 LEADER]: Becoming Leader. State: Replica: b1f31aafad064cc997b46dfd9b805d96, State: Running, Role: LEADER
I20260807 08:50:05.246253 25457 consensus_queue.cc:237] T 00000000000000000000000000000000 P b1f31aafad064cc997b46dfd9b805d96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1f31aafad064cc997b46dfd9b805d96" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 43597 } }
I20260807 08:50:05.246639 25460 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1f31aafad064cc997b46dfd9b805d96 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b1f31aafad064cc997b46dfd9b805d96" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1f31aafad064cc997b46dfd9b805d96" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 43597 } } }
I20260807 08:50:05.246771 25460 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1f31aafad064cc997b46dfd9b805d96 [sys.catalog]: This master's current role is: LEADER
I20260807 08:50:05.246745 25461 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1f31aafad064cc997b46dfd9b805d96 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b1f31aafad064cc997b46dfd9b805d96. Latest consensus state: current_term: 1 leader_uuid: "b1f31aafad064cc997b46dfd9b805d96" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1f31aafad064cc997b46dfd9b805d96" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 43597 } } }
I20260807 08:50:05.246865 25461 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1f31aafad064cc997b46dfd9b805d96 [sys.catalog]: This master's current role is: LEADER
I20260807 08:50:05.247409 25476 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:50:05.247830 25476 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:50:05.248932 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 25351
I20260807 08:50:05.249033 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786092453486433-4734-0/minicluster-data/master-0/wal/instance
I20260807 08:50:05.249243 4734 external_mini_cluster.cc:1428] Setting key 42c695c3e310167ed8a87434fb5a7a9b
I20260807 08:50:05.249271 25476 catalog_manager.cc:1347] Generated new cluster ID: 95fd0dea0ec9445faa65cb38cc4a1eb5
I20260807 08:50:05.249307 25476 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 08:50:05 dist-test-slave-6zbq krb5kdc[25335](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092605, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.159.190@KRBTEST.COM
I20260807 08:50:05.260558 25490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:05.249647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60152 (local address 127.4.159.190:43597)
0807 08:50:05.249772 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0807 08:50:05.249781 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 08:50:05.250525 (+ 744us) server_negotiation.cc:407] Connection header received
0807 08:50:05.250676 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:50:05.250683 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:50:05.250742 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:50:05.250873 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:50:05.251625 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:05.252956 (+ 1331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:05.254007 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:05.254603 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:05.254693 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:50:05.257225 (+ 2532us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:50:05.257229 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:50:05.257230 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:50:05.257249 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:50:05.259066 (+ 1817us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:50:05.259421 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:50:05.259423 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:50:05.259424 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:50:05.259523 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:50:05.259687 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:50:05.259688 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:50:05.259688 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:50:05.259968 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:50:05.260095 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0807 08:50:05.260193 (+ 98us) server_negotiation.cc:299] Negotiation successful
0807 08:50:05.260330 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93,"thread_start_us":54,"threads_started":1}
I20260807 08:50:05.261641 4734 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 08:50:05.270489 25476 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:50:05.271147 25476 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:50:05.277608 25476 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b1f31aafad064cc997b46dfd9b805d96: Generated new TSK 0
I20260807 08:50:05.277757 25476 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:50:05.315467 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 25351
2026-08-07T08:50:05Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1431 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:50:05 dist-test-slave-6zbq krb5kdc[25496](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:50:05 dist-test-slave-6zbq krb5kdc[25496](info): set up 2 sockets
Aug 07 08:50:05 dist-test-slave-6zbq krb5kdc[25496](info): commencing operation
krb5kdc: starting...
W20260807 08:50:06.437516 4734 mini_kdc.cc:121] Time spent starting KDC: real 1.094s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/krb5kdc/test-user.keytab.
Aug 07 08:50:06 dist-test-slave-6zbq krb5kdc[25496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092606, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:50:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:50:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.159.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.159.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.159.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab.
Entry for principal kudu/127.4.159.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab.
I20260807 08:50:06.545800 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.159.190:34273
--webserver_interface=127.4.159.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab
--principal=kudu/127.4.159.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:35687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.159.190:34273
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:50:06.627560 25514 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:50:06.627754 25514 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:50:06.627784 25514 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:50:06.628883 25514 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:50:06.628921 25514 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:50:06.628939 25514 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:50:06.628954 25514 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:50:06.628971 25514 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:50:06.630450 25514 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:35687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.159.190:34273
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab
--principal=kudu/127.4.159.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.159.190:34273
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.159.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25514
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:50:06.630674 25514 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:50:06.630862 25514 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:50:06.635268 25526 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:50:06.635243 25520 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:50:06.635253 25519 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:50:06.635457 25514 server_base.cc:1034] running on GCE node
I20260807 08:50:06.635667 25514 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:50:06.635929 25514 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:50:06.637089 25514 hybrid_clock.cc:648] HybridClock initialized: now 1786092606637060 us; error 31 us; skew 500 ppm
Aug 07 08:50:06 dist-test-slave-6zbq krb5kdc[25496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092606, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:50:06.639381 25514 init.cc:377] Logged in from keytab as kudu/127.4.159.190@KRBTEST.COM (short username kudu)
I20260807 08:50:06.639772 25514 webserver.cc:460] Webserver started at http://127.4.159.190:46171/ using document root <none> and password file <none>
I20260807 08:50:06.639957 25514 fs_manager.cc:359] Metadata directory not provided
I20260807 08:50:06.640002 25514 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:50:06.640120 25514 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:50:06.641098 25514 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/master-0/data/instance:
uuid: "7253adae706a42309f5b712541c5b57d"
format_stamp: "Formatted at 2026-08-07 08:50:06 on dist-test-slave-6zbq"
server_key: "a8bb7a8ab5a147d14a7be7e4363c27a0"
server_key_iv: "e8133f3cd3a115d8ba6b2db3505a635a"
server_key_version: "encryptionkey@0"
I20260807 08:50:06.641430 25514 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/master-0/wal/instance:
uuid: "7253adae706a42309f5b712541c5b57d"
format_stamp: "Formatted at 2026-08-07 08:50:06 on dist-test-slave-6zbq"
server_key: "a8bb7a8ab5a147d14a7be7e4363c27a0"
server_key_iv: "e8133f3cd3a115d8ba6b2db3505a635a"
server_key_version: "encryptionkey@0"
I20260807 08:50:06.642635 25514 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 08:50:06.643271 25536 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:50:06.643417 25514 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 08:50:06.643496 25514 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/master-0/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/master-0/wal
uuid: "7253adae706a42309f5b712541c5b57d"
format_stamp: "Formatted at 2026-08-07 08:50:06 on dist-test-slave-6zbq"
server_key: "a8bb7a8ab5a147d14a7be7e4363c27a0"
server_key_iv: "e8133f3cd3a115d8ba6b2db3505a635a"
server_key_version: "encryptionkey@0"
I20260807 08:50:06.643548 25514 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:50:06.653903 25514 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:50:06.655755 25514 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:50:06.655853 25514 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:50:06.659447 25514 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.190:34273
I20260807 08:50:06.659479 25616 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.190:34273 every 8 connection(s)
I20260807 08:50:06.659807 25514 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
I20260807 08:50:06.660728 25617 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:50:06.662976 25617 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7253adae706a42309f5b712541c5b57d: Bootstrap starting.
I20260807 08:50:06.663467 25617 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7253adae706a42309f5b712541c5b57d: Neither blocks nor log segments found. Creating new log.
I20260807 08:50:06.663702 25617 log.cc:824] T 00000000000000000000000000000000 P 7253adae706a42309f5b712541c5b57d: Log is configured to *not* fsync() on all Append() calls
I20260807 08:50:06.664299 25617 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7253adae706a42309f5b712541c5b57d: No bootstrap required, opened a new log
I20260807 08:50:06.665818 25617 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7253adae706a42309f5b712541c5b57d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7253adae706a42309f5b712541c5b57d" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 34273 } }
I20260807 08:50:06.665943 25617 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7253adae706a42309f5b712541c5b57d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:50:06.665970 25617 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7253adae706a42309f5b712541c5b57d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7253adae706a42309f5b712541c5b57d, State: Initialized, Role: FOLLOWER
I20260807 08:50:06.666067 25617 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7253adae706a42309f5b712541c5b57d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7253adae706a42309f5b712541c5b57d" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 34273 } }
I20260807 08:50:06.666141 25617 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7253adae706a42309f5b712541c5b57d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:50:06.666167 25617 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7253adae706a42309f5b712541c5b57d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:50:06.666198 25617 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7253adae706a42309f5b712541c5b57d [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:50:06.666952 25617 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7253adae706a42309f5b712541c5b57d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7253adae706a42309f5b712541c5b57d" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 34273 } }
I20260807 08:50:06.667037 25617 leader_election.cc:302] T 00000000000000000000000000000000 P 7253adae706a42309f5b712541c5b57d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7253adae706a42309f5b712541c5b57d; no voters:
I20260807 08:50:06.667147 25617 leader_election.cc:288] T 00000000000000000000000000000000 P 7253adae706a42309f5b712541c5b57d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:50:06.667179 25623 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7253adae706a42309f5b712541c5b57d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:50:06.667348 25617 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7253adae706a42309f5b712541c5b57d [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:50:06.667347 25623 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7253adae706a42309f5b712541c5b57d [term 1 LEADER]: Becoming Leader. State: Replica: 7253adae706a42309f5b712541c5b57d, State: Running, Role: LEADER
I20260807 08:50:06.667526 25623 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7253adae706a42309f5b712541c5b57d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7253adae706a42309f5b712541c5b57d" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 34273 } }
I20260807 08:50:06.667886 25624 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7253adae706a42309f5b712541c5b57d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7253adae706a42309f5b712541c5b57d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7253adae706a42309f5b712541c5b57d" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 34273 } } }
I20260807 08:50:06.667968 25624 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7253adae706a42309f5b712541c5b57d [sys.catalog]: This master's current role is: LEADER
I20260807 08:50:06.667869 25627 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7253adae706a42309f5b712541c5b57d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7253adae706a42309f5b712541c5b57d. Latest consensus state: current_term: 1 leader_uuid: "7253adae706a42309f5b712541c5b57d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7253adae706a42309f5b712541c5b57d" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 34273 } } }
I20260807 08:50:06.668125 25627 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7253adae706a42309f5b712541c5b57d [sys.catalog]: This master's current role is: LEADER
I20260807 08:50:06.668227 25636 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:50:06.668676 25636 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:50:06.669945 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 25514
I20260807 08:50:06.670058 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/master-0/wal/instance
I20260807 08:50:06.670250 4734 external_mini_cluster.cc:1428] Setting key 829150a09f8b6dfb6051cdce1c160d8a
I20260807 08:50:06.670432 25636 catalog_manager.cc:1347] Generated new cluster ID: e34fe26867a9489194587342451da3f2
I20260807 08:50:06.670477 25636 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 08:50:06 dist-test-slave-6zbq krb5kdc[25496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092606, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.159.190@KRBTEST.COM
I20260807 08:50:06.684538 25654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:06.670653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59530 (local address 127.4.159.190:34273)
0807 08:50:06.674652 (+ 3999us) server_negotiation.cc:206] Beginning negotiation
0807 08:50:06.674661 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 08:50:06.674692 (+ 31us) server_negotiation.cc:407] Connection header received
0807 08:50:06.674802 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:50:06.674809 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:50:06.674861 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:50:06.674988 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:50:06.675655 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:06.677063 (+ 1408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:06.678026 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:06.678657 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:06.678755 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:50:06.681292 (+ 2537us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:50:06.681296 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:50:06.681297 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:50:06.681307 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:50:06.683070 (+ 1763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:50:06.683447 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:50:06.683450 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:50:06.683450 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:50:06.683539 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:50:06.683713 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:50:06.683714 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:50:06.683714 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:50:06.683974 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:50:06.684111 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0807 08:50:06.684218 (+ 107us) server_negotiation.cc:299] Negotiation successful
0807 08:50:06.684338 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3955,"thread_start_us":66,"threads_started":1}
I20260807 08:50:06.693220 25636 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:50:06.693914 25636 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:50:06.699054 25636 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7253adae706a42309f5b712541c5b57d: Generated new TSK 0
I20260807 08:50:06.699199 25636 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.159.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.159.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.159.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.129.keytab.
Entry for principal kudu/127.4.159.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.129.keytab.
I20260807 08:50:06.723014 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.159.129:0
--local_ip_for_outbound_sockets=127.4.159.129
--webserver_interface=127.4.159.129
--webserver_port=0
--tserver_master_addrs=127.4.159.190:34273
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.129.keytab
--principal=kudu/127.4.159.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:35687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:50:06.801867 25658 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:50:06.802040 25658 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:50:06.802060 25658 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:50:06.803115 25658 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:50:06.803149 25658 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:50:06.803174 25658 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.159.129
I20260807 08:50:06.804545 25658 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:35687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.129.keytab
--principal=kudu/127.4.159.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.159.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.159.129
--webserver_port=0
--tserver_master_addrs=127.4.159.190:34273
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25658
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.159.129
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:50:06.804725 25658 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:50:06.804899 25658 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:50:06.809643 25664 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:50:06.809710 25658 server_base.cc:1034] running on GCE node
W20260807 08:50:06.809666 25663 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:50:06.809680 25666 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:50:06.809980 25658 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:50:06.810185 25658 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:50:06.811337 25658 hybrid_clock.cc:648] HybridClock initialized: now 1786092606811324 us; error 25 us; skew 500 ppm
Aug 07 08:50:06 dist-test-slave-6zbq krb5kdc[25496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092606, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:50:06.813508 25658 init.cc:377] Logged in from keytab as kudu/127.4.159.129@KRBTEST.COM (short username kudu)
I20260807 08:50:06.813898 25658 webserver.cc:460] Webserver started at http://127.4.159.129:41265/ using document root <none> and password file <none>
I20260807 08:50:06.814105 25658 fs_manager.cc:359] Metadata directory not provided
I20260807 08:50:06.814150 25658 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:50:06.814273 25658 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:50:06.815322 25658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-0/data/instance:
uuid: "9181e852d19c47d2bb93dec238a89ffb"
format_stamp: "Formatted at 2026-08-07 08:50:06 on dist-test-slave-6zbq"
server_key: "f0e514983f7d1a8dc026d8457e216f4a"
server_key_iv: "7858241c60bdeb4e4b048a12f6a3c95b"
server_key_version: "encryptionkey@0"
I20260807 08:50:06.815627 25658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-0/wal/instance:
uuid: "9181e852d19c47d2bb93dec238a89ffb"
format_stamp: "Formatted at 2026-08-07 08:50:06 on dist-test-slave-6zbq"
server_key: "f0e514983f7d1a8dc026d8457e216f4a"
server_key_iv: "7858241c60bdeb4e4b048a12f6a3c95b"
server_key_version: "encryptionkey@0"
I20260807 08:50:06.816794 25658 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 08:50:06.817423 25674 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:50:06.817565 25658 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 08:50:06.817643 25658 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-0/wal
uuid: "9181e852d19c47d2bb93dec238a89ffb"
format_stamp: "Formatted at 2026-08-07 08:50:06 on dist-test-slave-6zbq"
server_key: "f0e514983f7d1a8dc026d8457e216f4a"
server_key_iv: "7858241c60bdeb4e4b048a12f6a3c95b"
server_key_version: "encryptionkey@0"
I20260807 08:50:06.817709 25658 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:50:06.830950 25658 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:50:06.832505 25658 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:50:06.832623 25658 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:50:06.832800 25658 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:50:06.833055 25658 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:50:06.833086 25658 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:50:06.833110 25658 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:50:06.833128 25658 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:50:06.838456 25658 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.129:46023
I20260807 08:50:06.838492 25849 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.129:46023 every 8 connection(s)
I20260807 08:50:06.838745 25658 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-0/data/info.pb
Aug 07 08:50:06 dist-test-slave-6zbq krb5kdc[25496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092606, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.129@KRBTEST.COM for kudu/127.4.159.190@KRBTEST.COM
I20260807 08:50:06.846489 25654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:06.839339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.129:52137 (local address 127.4.159.190:34273)
0807 08:50:06.839390 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0807 08:50:06.839391 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 08:50:06.839578 (+ 187us) server_negotiation.cc:407] Connection header received
0807 08:50:06.839998 (+ 420us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:50:06.839999 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:50:06.840013 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:50:06.840079 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:50:06.840913 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:06.841403 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:06.842277 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:06.842508 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:06.842551 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:50:06.844380 (+ 1829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:50:06.844382 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:50:06.844382 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:50:06.844389 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:50:06.845416 (+ 1027us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:50:06.845759 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:50:06.845760 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:50:06.845760 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:50:06.845801 (+ 41us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:50:06.845984 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:50:06.845984 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:50:06.845985 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:50:06.846080 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:50:06.846127 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0807 08:50:06.846331 (+ 204us) server_negotiation.cc:299] Negotiation successful
0807 08:50:06.846426 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260807 08:50:06.846635 25856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:06.839316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:34273 (local address 127.4.159.129:52137)
0807 08:50:06.839469 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:06.839508 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 08:50:06.839925 (+ 417us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:50:06.840126 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:50:06.840128 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:50:06.840621 (+ 493us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:50:06.840802 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:50:06.840806 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:50:06.841468 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:50:06.841469 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:50:06.842220 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:50:06.842222 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:50:06.842564 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:50:06.842565 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:50:06.842652 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:50:06.843068 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:50:06.843076 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:50:06.844293 (+ 1217us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:50:06.845486 (+ 1193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:50:06.845493 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:50:06.845495 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:50:06.845695 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:50:06.845847 (+ 152us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:50:06.845847 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:50:06.845847 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:50:06.845939 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:50:06.846129 (+ 190us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:50:06.846133 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:50:06.846233 (+ 100us) client_negotiation.cc:769] Sending connection context
0807 08:50:06.846315 (+ 82us) client_negotiation.cc:240] Negotiation successful
0807 08:50:06.846460 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":77,"threads_started":1}
I20260807 08:50:06.847065 25850 heartbeater.cc:347] Connected to a master server at 127.4.159.190:34273
I20260807 08:50:06.847146 25850 heartbeater.cc:464] Registering TS with master...
I20260807 08:50:06.847165 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 25658
I20260807 08:50:06.847257 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-0/wal/instance
I20260807 08:50:06.847327 25850 heartbeater.cc:511] Master 127.4.159.190:34273 requested a full tablet report, sending...
I20260807 08:50:06.847455 4734 external_mini_cluster.cc:1428] Setting key dacf3eb2155730a7ea0cf26f540b4560
I20260807 08:50:06.847841 25564 ts_manager.cc:194] Registered new tserver with Master: 9181e852d19c47d2bb93dec238a89ffb (127.4.159.129:46023)
I20260807 08:50:06.848496 25564 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.159.129@KRBTEST.COM'} at 127.4.159.129:52137
WARNING: no policy specified for kudu/127.4.159.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.159.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.159.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.130.keytab.
Entry for principal kudu/127.4.159.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.130.keytab.
I20260807 08:50:06.870774 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.159.130:0
--local_ip_for_outbound_sockets=127.4.159.130
--webserver_interface=127.4.159.130
--webserver_port=0
--tserver_master_addrs=127.4.159.190:34273
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.130.keytab
--principal=kudu/127.4.159.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:35687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:50:06.949620 25862 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:50:06.949800 25862 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:50:06.949831 25862 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:50:06.950987 25862 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:50:06.951028 25862 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:50:06.951062 25862 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.159.130
I20260807 08:50:06.952461 25862 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:35687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.130.keytab
--principal=kudu/127.4.159.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.159.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.159.130
--webserver_port=0
--tserver_master_addrs=127.4.159.190:34273
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25862
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.159.130
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:50:06.952672 25862 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:50:06.952867 25862 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:50:06.957338 25870 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:50:06.957351 25876 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:50:06.957356 25871 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:50:06.957509 25862 server_base.cc:1034] running on GCE node
I20260807 08:50:06.957677 25862 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:50:06.957863 25862 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:50:06.958993 25862 hybrid_clock.cc:648] HybridClock initialized: now 1786092606958979 us; error 28 us; skew 500 ppm
Aug 07 08:50:06 dist-test-slave-6zbq krb5kdc[25496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092606, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:50:06.961274 25862 init.cc:377] Logged in from keytab as kudu/127.4.159.130@KRBTEST.COM (short username kudu)
I20260807 08:50:06.961678 25862 webserver.cc:460] Webserver started at http://127.4.159.130:38387/ using document root <none> and password file <none>
I20260807 08:50:06.961856 25862 fs_manager.cc:359] Metadata directory not provided
I20260807 08:50:06.961902 25862 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:50:06.962021 25862 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:50:06.962996 25862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-1/data/instance:
uuid: "dccc17259c6e4d35bcf60aae6b6cdfe4"
format_stamp: "Formatted at 2026-08-07 08:50:06 on dist-test-slave-6zbq"
server_key: "f0d427e8337b42861d2f308790acd1f1"
server_key_iv: "d4f7e16f06cac3bc5945628c3ad319eb"
server_key_version: "encryptionkey@0"
I20260807 08:50:06.963317 25862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-1/wal/instance:
uuid: "dccc17259c6e4d35bcf60aae6b6cdfe4"
format_stamp: "Formatted at 2026-08-07 08:50:06 on dist-test-slave-6zbq"
server_key: "f0d427e8337b42861d2f308790acd1f1"
server_key_iv: "d4f7e16f06cac3bc5945628c3ad319eb"
server_key_version: "encryptionkey@0"
I20260807 08:50:06.964520 25862 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 08:50:06.965166 25884 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:50:06.965312 25862 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 08:50:06.965392 25862 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-1/wal
uuid: "dccc17259c6e4d35bcf60aae6b6cdfe4"
format_stamp: "Formatted at 2026-08-07 08:50:06 on dist-test-slave-6zbq"
server_key: "f0d427e8337b42861d2f308790acd1f1"
server_key_iv: "d4f7e16f06cac3bc5945628c3ad319eb"
server_key_version: "encryptionkey@0"
I20260807 08:50:06.965451 25862 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:50:06.979408 25862 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:50:06.981379 25862 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:50:06.981504 25862 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:50:06.981693 25862 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:50:06.981954 25862 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:50:06.981987 25862 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:50:06.982013 25862 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:50:06.982030 25862 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:50:06.987286 25862 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.130:34071
I20260807 08:50:06.987351 26049 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.130:34071 every 8 connection(s)
I20260807 08:50:06.987605 25862 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-1/data/info.pb
Aug 07 08:50:06 dist-test-slave-6zbq krb5kdc[25496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092606, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.130@KRBTEST.COM for kudu/127.4.159.190@KRBTEST.COM
I20260807 08:50:06.994866 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 25862
I20260807 08:50:06.994971 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-1/wal/instance
I20260807 08:50:06.995167 4734 external_mini_cluster.cc:1428] Setting key dafe0dc2195168ac37051aadba86fbdb
I20260807 08:50:06.995627 25654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:06.988201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.130:42033 (local address 127.4.159.190:34273)
0807 08:50:06.988261 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0807 08:50:06.988262 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 08:50:06.988408 (+ 146us) server_negotiation.cc:407] Connection header received
0807 08:50:06.988822 (+ 414us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:50:06.988823 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:50:06.988832 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:50:06.988882 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:50:06.989697 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:06.990173 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:06.991087 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:06.991262 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:06.991313 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:50:06.993143 (+ 1830us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:50:06.993144 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:50:06.993144 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:50:06.993151 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:50:06.994331 (+ 1180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:50:06.994718 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:50:06.994719 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:50:06.994719 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:50:06.994773 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:50:06.994988 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:50:06.994989 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:50:06.994989 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:50:06.995119 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:50:06.995175 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 08:50:06.995456 (+ 281us) server_negotiation.cc:299] Negotiation successful
0807 08:50:06.995539 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260807 08:50:06.995765 26053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:06.988153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:34273 (local address 127.4.159.130:42033)
0807 08:50:06.988300 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:06.988336 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 08:50:06.988744 (+ 408us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:50:06.988930 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:50:06.988933 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:50:06.989431 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:50:06.989599 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:50:06.989603 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:50:06.990246 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:50:06.990246 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:50:06.991027 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:50:06.991030 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:50:06.991317 (+ 287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:50:06.991317 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:50:06.991405 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:50:06.991753 (+ 348us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:50:06.991760 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:50:06.993036 (+ 1276us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:50:06.994428 (+ 1392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:50:06.994435 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:50:06.994437 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:50:06.994652 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:50:06.994829 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:50:06.994830 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:50:06.994830 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:50:06.994931 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:50:06.995178 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:50:06.995186 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:50:06.995335 (+ 149us) client_negotiation.cc:769] Sending connection context
0807 08:50:06.995422 (+ 87us) client_negotiation.cc:240] Negotiation successful
0807 08:50:06.995561 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":83,"threads_started":1}
I20260807 08:50:06.996166 26050 heartbeater.cc:347] Connected to a master server at 127.4.159.190:34273
I20260807 08:50:06.996253 26050 heartbeater.cc:464] Registering TS with master...
I20260807 08:50:06.996408 26050 heartbeater.cc:511] Master 127.4.159.190:34273 requested a full tablet report, sending...
I20260807 08:50:06.996791 25564 ts_manager.cc:194] Registered new tserver with Master: dccc17259c6e4d35bcf60aae6b6cdfe4 (127.4.159.130:34071)
I20260807 08:50:06.997162 25564 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.159.130@KRBTEST.COM'} at 127.4.159.130:42033
WARNING: no policy specified for kudu/127.4.159.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.159.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.159.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.131.keytab.
Entry for principal kudu/127.4.159.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.131.keytab.
I20260807 08:50:07.018617 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.159.131:0
--local_ip_for_outbound_sockets=127.4.159.131
--webserver_interface=127.4.159.131
--webserver_port=0
--tserver_master_addrs=127.4.159.190:34273
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.131.keytab
--principal=kudu/127.4.159.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:35687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:50:07.097146 26056 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:50:07.097335 26056 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:50:07.097365 26056 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:50:07.098498 26056 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:50:07.098537 26056 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:50:07.098573 26056 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.159.131
I20260807 08:50:07.099957 26056 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:35687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.131.keytab
--principal=kudu/127.4.159.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.159.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.159.131
--webserver_port=0
--tserver_master_addrs=127.4.159.190:34273
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26056
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.159.131
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:50:07.100167 26056 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:50:07.100365 26056 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:50:07.105029 26067 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:50:07.105082 26065 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:50:07.105031 26063 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:50:07.105240 26056 server_base.cc:1034] running on GCE node
I20260807 08:50:07.105396 26056 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:50:07.105571 26056 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:50:07.106705 26056 hybrid_clock.cc:648] HybridClock initialized: now 1786092607106693 us; error 26 us; skew 500 ppm
Aug 07 08:50:07 dist-test-slave-6zbq krb5kdc[25496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092607, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:50:07.109200 26056 init.cc:377] Logged in from keytab as kudu/127.4.159.131@KRBTEST.COM (short username kudu)
I20260807 08:50:07.109604 26056 webserver.cc:460] Webserver started at http://127.4.159.131:45051/ using document root <none> and password file <none>
I20260807 08:50:07.109793 26056 fs_manager.cc:359] Metadata directory not provided
I20260807 08:50:07.109834 26056 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:50:07.109954 26056 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:50:07.110956 26056 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-2/data/instance:
uuid: "e65182db1a254905939ae7ae864acfd7"
format_stamp: "Formatted at 2026-08-07 08:50:07 on dist-test-slave-6zbq"
server_key: "9ef9a62a6ed0d08b081ae4530045c96c"
server_key_iv: "db5536153bf901ec0f6dc22acf5ad28e"
server_key_version: "encryptionkey@0"
I20260807 08:50:07.111267 26056 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-2/wal/instance:
uuid: "e65182db1a254905939ae7ae864acfd7"
format_stamp: "Formatted at 2026-08-07 08:50:07 on dist-test-slave-6zbq"
server_key: "9ef9a62a6ed0d08b081ae4530045c96c"
server_key_iv: "db5536153bf901ec0f6dc22acf5ad28e"
server_key_version: "encryptionkey@0"
I20260807 08:50:07.112437 26056 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 08:50:07.113060 26077 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:50:07.113199 26056 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 08:50:07.113274 26056 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-2/wal
uuid: "e65182db1a254905939ae7ae864acfd7"
format_stamp: "Formatted at 2026-08-07 08:50:07 on dist-test-slave-6zbq"
server_key: "9ef9a62a6ed0d08b081ae4530045c96c"
server_key_iv: "db5536153bf901ec0f6dc22acf5ad28e"
server_key_version: "encryptionkey@0"
I20260807 08:50:07.113336 26056 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:50:07.145562 26056 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:50:07.147253 26056 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:50:07.147368 26056 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:50:07.147565 26056 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:50:07.147833 26056 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:50:07.147864 26056 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:50:07.147882 26056 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:50:07.147905 26056 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:50:07.153163 26056 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.131:34385
I20260807 08:50:07.153198 26258 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.131:34385 every 8 connection(s)
I20260807 08:50:07.153450 26056 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-2/data/info.pb
Aug 07 08:50:07 dist-test-slave-6zbq krb5kdc[25496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092607, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.131@KRBTEST.COM for kudu/127.4.159.190@KRBTEST.COM
I20260807 08:50:07.161305 25654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:07.154091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.131:59229 (local address 127.4.159.190:34273)
0807 08:50:07.154140 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0807 08:50:07.154142 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:50:07.154300 (+ 158us) server_negotiation.cc:407] Connection header received
0807 08:50:07.154775 (+ 475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:50:07.154776 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:50:07.154787 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:50:07.154825 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:50:07.155606 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:07.156084 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:07.156965 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:07.157149 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:07.157190 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:50:07.159031 (+ 1841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:50:07.159033 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:50:07.159033 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:50:07.159041 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:50:07.160084 (+ 1043us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:50:07.160426 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:50:07.160427 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:50:07.160427 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:50:07.160482 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:50:07.160705 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:50:07.160706 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:50:07.160707 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:50:07.160826 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:50:07.160883 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 08:50:07.161127 (+ 244us) server_negotiation.cc:299] Negotiation successful
0807 08:50:07.161217 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260807 08:50:07.161433 26262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:07.154026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:34273 (local address 127.4.159.131:59229)
0807 08:50:07.154182 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:07.154225 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 08:50:07.154680 (+ 455us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:50:07.154874 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:50:07.154878 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:50:07.155378 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:50:07.155543 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:50:07.155548 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:50:07.156141 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:50:07.156142 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:50:07.156904 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:50:07.156906 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:50:07.157198 (+ 292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:50:07.157199 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:50:07.157285 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:50:07.157681 (+ 396us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:50:07.157688 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:50:07.158945 (+ 1257us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:50:07.160151 (+ 1206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:50:07.160157 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:50:07.160159 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:50:07.160361 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:50:07.160535 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:50:07.160536 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:50:07.160536 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:50:07.160655 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:50:07.160880 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:50:07.160885 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:50:07.161003 (+ 118us) client_negotiation.cc:769] Sending connection context
0807 08:50:07.161096 (+ 93us) client_negotiation.cc:240] Negotiation successful
0807 08:50:07.161232 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":77,"threads_started":1}
I20260807 08:50:07.161832 26259 heartbeater.cc:347] Connected to a master server at 127.4.159.190:34273
I20260807 08:50:07.161913 26259 heartbeater.cc:464] Registering TS with master...
I20260807 08:50:07.162060 26259 heartbeater.cc:511] Master 127.4.159.190:34273 requested a full tablet report, sending...
I20260807 08:50:07.162405 25564 ts_manager.cc:194] Registered new tserver with Master: e65182db1a254905939ae7ae864acfd7 (127.4.159.131:34385)
I20260807 08:50:07.162761 25564 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.159.131@KRBTEST.COM'} at 127.4.159.131:59229
I20260807 08:50:07.163125 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 26056
I20260807 08:50:07.163228 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-2/wal/instance
I20260807 08:50:07.163436 4734 external_mini_cluster.cc:1428] Setting key b4d38c0044fafaa12230ce792a6fe346
I20260807 08:50:07.164191 4734 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 08:50:07.171029 25654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:07.165384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59544 (local address 127.4.159.190:34273)
0807 08:50:07.165441 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0807 08:50:07.165442 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 08:50:07.165498 (+ 56us) server_negotiation.cc:407] Connection header received
0807 08:50:07.165553 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:50:07.165554 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:50:07.165561 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:50:07.165595 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:50:07.166074 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:07.166557 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:07.167227 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:07.167450 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:07.167494 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:50:07.168636 (+ 1142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:50:07.168637 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:50:07.168637 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:50:07.168643 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:50:07.169806 (+ 1163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:50:07.170444 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:50:07.170445 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:50:07.170446 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:50:07.170501 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:50:07.170648 (+ 147us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:50:07.170648 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:50:07.170649 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:50:07.170771 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:50:07.170837 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 08:50:07.170914 (+ 77us) server_negotiation.cc:299] Negotiation successful
0807 08:50:07.170971 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260807 08:50:07.172449 25564 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59544:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 08:50:07.173918 25564 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:50:07.179808 26281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:07.176091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60244 (local address 127.4.159.129:46023)
0807 08:50:07.176250 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0807 08:50:07.176254 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:50:07.176341 (+ 87us) server_negotiation.cc:407] Connection header received
0807 08:50:07.176416 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:50:07.176418 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:50:07.176435 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:50:07.176504 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:50:07.176708 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:07.177849 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:07.178956 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:07.179433 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:07.179497 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:50:07.179551 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 08:50:07.179686 (+ 135us) server_negotiation.cc:299] Negotiation successful
0807 08:50:07.179728 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":85,"threads_started":1}
I20260807 08:50:07.179826 26277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:07.175855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.129:46023 (local address 127.0.0.1:60244)
0807 08:50:07.176277 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:07.176286 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 08:50:07.176341 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:50:07.176587 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:50:07.176589 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:50:07.176591 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:50:07.176635 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:50:07.176637 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:50:07.177949 (+ 1312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:50:07.177950 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:50:07.178883 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:50:07.178885 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:50:07.179508 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:50:07.179509 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:50:07.179587 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:50:07.179600 (+ 13us) client_negotiation.cc:769] Sending connection context
0807 08:50:07.179663 (+ 63us) client_negotiation.cc:240] Negotiation successful
0807 08:50:07.179714 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":409,"mutex_wait_us":34,"thread_start_us":59,"threads_started":1}
I20260807 08:50:07.180035 26278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:07.175951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37402 (local address 127.4.159.130:34071)
0807 08:50:07.176133 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0807 08:50:07.176137 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:50:07.176308 (+ 171us) server_negotiation.cc:407] Connection header received
0807 08:50:07.176344 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:50:07.176346 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:50:07.176358 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:50:07.176418 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:50:07.176565 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:07.177408 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:07.178712 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:07.179434 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:07.179496 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:50:07.179547 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0807 08:50:07.179909 (+ 362us) server_negotiation.cc:299] Negotiation successful
0807 08:50:07.179955 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":85,"threads_started":1}
I20260807 08:50:07.179839 26275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:07.175846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.130:34071 (local address 127.0.0.1:37402)
0807 08:50:07.176133 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:07.176174 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 08:50:07.176245 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:50:07.176439 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:50:07.176442 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:50:07.176444 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:50:07.176506 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:50:07.176508 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:50:07.177503 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:50:07.177504 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:50:07.178617 (+ 1113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:50:07.178620 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:50:07.179581 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:50:07.179582 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:50:07.179643 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:50:07.179648 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 08:50:07.179695 (+ 47us) client_negotiation.cc:240] Negotiation successful
0807 08:50:07.179743 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"mutex_wait_us":1,"thread_start_us":55,"threads_started":1}
I20260807 08:50:07.180233 26276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:07.175857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.131:34385 (local address 127.0.0.1:40812)
0807 08:50:07.176417 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:07.176425 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0807 08:50:07.176466 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:50:07.176697 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:50:07.176698 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:50:07.176700 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:50:07.176729 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:50:07.176730 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:50:07.177940 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:50:07.177941 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:50:07.179068 (+ 1127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:50:07.179071 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:50:07.179971 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:50:07.179972 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:50:07.180033 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:50:07.180037 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 08:50:07.180126 (+ 89us) client_negotiation.cc:240] Negotiation successful
0807 08:50:07.180168 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":551,"mutex_wait_us":28,"spinlock_wait_cycles":384,"thread_start_us":60,"threads_started":1}
I20260807 08:50:07.180709 25747 tablet_service.cc:1467] Processing CreateTablet for tablet 5656d45db93f4f3ab57832ad63ca399e (DEFAULT_TABLE table=test-table [id=22d6560a2be64420bba17c5a1de9ed50]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:50:07.180730 25963 tablet_service.cc:1467] Processing CreateTablet for tablet 5656d45db93f4f3ab57832ad63ca399e (DEFAULT_TABLE table=test-table [id=22d6560a2be64420bba17c5a1de9ed50]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:50:07.181057 25747 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5656d45db93f4f3ab57832ad63ca399e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:50:07.181057 25963 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5656d45db93f4f3ab57832ad63ca399e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:50:07.182495 26279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:07.176046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40812 (local address 127.4.159.131:34385)
0807 08:50:07.176200 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0807 08:50:07.176204 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:50:07.176520 (+ 316us) server_negotiation.cc:407] Connection header received
0807 08:50:07.176557 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:50:07.176559 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:50:07.176572 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:50:07.176633 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:50:07.176798 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:07.177833 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:07.179156 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:07.179829 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:07.179888 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:50:07.179939 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0807 08:50:07.182403 (+ 2464us) server_negotiation.cc:299] Negotiation successful
0807 08:50:07.182439 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":60,"threads_started":1}
I20260807 08:50:07.184345 26284 tablet_bootstrap.cc:492] T 5656d45db93f4f3ab57832ad63ca399e P 9181e852d19c47d2bb93dec238a89ffb: Bootstrap starting.
I20260807 08:50:07.184804 26155 tablet_service.cc:1467] Processing CreateTablet for tablet 5656d45db93f4f3ab57832ad63ca399e (DEFAULT_TABLE table=test-table [id=22d6560a2be64420bba17c5a1de9ed50]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:50:07.185072 26155 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5656d45db93f4f3ab57832ad63ca399e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:50:07.185078 26284 tablet_bootstrap.cc:654] T 5656d45db93f4f3ab57832ad63ca399e P 9181e852d19c47d2bb93dec238a89ffb: Neither blocks nor log segments found. Creating new log.
I20260807 08:50:07.185539 26284 log.cc:824] T 5656d45db93f4f3ab57832ad63ca399e P 9181e852d19c47d2bb93dec238a89ffb: Log is configured to *not* fsync() on all Append() calls
I20260807 08:50:07.185820 26282 tablet_bootstrap.cc:492] T 5656d45db93f4f3ab57832ad63ca399e P dccc17259c6e4d35bcf60aae6b6cdfe4: Bootstrap starting.
I20260807 08:50:07.186236 26284 tablet_bootstrap.cc:492] T 5656d45db93f4f3ab57832ad63ca399e P 9181e852d19c47d2bb93dec238a89ffb: No bootstrap required, opened a new log
I20260807 08:50:07.186311 26284 ts_tablet_manager.cc:1397] T 5656d45db93f4f3ab57832ad63ca399e P 9181e852d19c47d2bb93dec238a89ffb: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 08:50:07.186311 26282 tablet_bootstrap.cc:654] T 5656d45db93f4f3ab57832ad63ca399e P dccc17259c6e4d35bcf60aae6b6cdfe4: Neither blocks nor log segments found. Creating new log.
I20260807 08:50:07.186782 26282 log.cc:824] T 5656d45db93f4f3ab57832ad63ca399e P dccc17259c6e4d35bcf60aae6b6cdfe4: Log is configured to *not* fsync() on all Append() calls
I20260807 08:50:07.187434 26282 tablet_bootstrap.cc:492] T 5656d45db93f4f3ab57832ad63ca399e P dccc17259c6e4d35bcf60aae6b6cdfe4: No bootstrap required, opened a new log
I20260807 08:50:07.187491 26282 ts_tablet_manager.cc:1397] T 5656d45db93f4f3ab57832ad63ca399e P dccc17259c6e4d35bcf60aae6b6cdfe4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 08:50:07.188047 26289 tablet_bootstrap.cc:492] T 5656d45db93f4f3ab57832ad63ca399e P e65182db1a254905939ae7ae864acfd7: Bootstrap starting.
I20260807 08:50:07.188221 26284 raft_consensus.cc:348] T 5656d45db93f4f3ab57832ad63ca399e P 9181e852d19c47d2bb93dec238a89ffb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dccc17259c6e4d35bcf60aae6b6cdfe4" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 34071 } } peers { permanent_uuid: "9181e852d19c47d2bb93dec238a89ffb" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 46023 } } peers { permanent_uuid: "e65182db1a254905939ae7ae864acfd7" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 34385 } }
I20260807 08:50:07.188364 26284 raft_consensus.cc:374] T 5656d45db93f4f3ab57832ad63ca399e P 9181e852d19c47d2bb93dec238a89ffb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:50:07.188395 26284 raft_consensus.cc:729] T 5656d45db93f4f3ab57832ad63ca399e P 9181e852d19c47d2bb93dec238a89ffb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9181e852d19c47d2bb93dec238a89ffb, State: Initialized, Role: FOLLOWER
I20260807 08:50:07.188524 26284 consensus_queue.cc:260] T 5656d45db93f4f3ab57832ad63ca399e P 9181e852d19c47d2bb93dec238a89ffb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dccc17259c6e4d35bcf60aae6b6cdfe4" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 34071 } } peers { permanent_uuid: "9181e852d19c47d2bb93dec238a89ffb" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 46023 } } peers { permanent_uuid: "e65182db1a254905939ae7ae864acfd7" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 34385 } }
I20260807 08:50:07.188776 26289 tablet_bootstrap.cc:654] T 5656d45db93f4f3ab57832ad63ca399e P e65182db1a254905939ae7ae864acfd7: Neither blocks nor log segments found. Creating new log.
I20260807 08:50:07.188795 26284 ts_tablet_manager.cc:1428] T 5656d45db93f4f3ab57832ad63ca399e P 9181e852d19c47d2bb93dec238a89ffb: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 08:50:07.188920 25850 heartbeater.cc:503] Master 127.4.159.190:34273 was elected leader, sending a full tablet report...
I20260807 08:50:07.189262 26289 log.cc:824] T 5656d45db93f4f3ab57832ad63ca399e P e65182db1a254905939ae7ae864acfd7: Log is configured to *not* fsync() on all Append() calls
I20260807 08:50:07.189239 26282 raft_consensus.cc:348] T 5656d45db93f4f3ab57832ad63ca399e P dccc17259c6e4d35bcf60aae6b6cdfe4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dccc17259c6e4d35bcf60aae6b6cdfe4" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 34071 } } peers { permanent_uuid: "9181e852d19c47d2bb93dec238a89ffb" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 46023 } } peers { permanent_uuid: "e65182db1a254905939ae7ae864acfd7" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 34385 } }
I20260807 08:50:07.189399 26282 raft_consensus.cc:374] T 5656d45db93f4f3ab57832ad63ca399e P dccc17259c6e4d35bcf60aae6b6cdfe4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:50:07.189442 26282 raft_consensus.cc:729] T 5656d45db93f4f3ab57832ad63ca399e P dccc17259c6e4d35bcf60aae6b6cdfe4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dccc17259c6e4d35bcf60aae6b6cdfe4, State: Initialized, Role: FOLLOWER
I20260807 08:50:07.189551 26282 consensus_queue.cc:260] T 5656d45db93f4f3ab57832ad63ca399e P dccc17259c6e4d35bcf60aae6b6cdfe4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dccc17259c6e4d35bcf60aae6b6cdfe4" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 34071 } } peers { permanent_uuid: "9181e852d19c47d2bb93dec238a89ffb" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 46023 } } peers { permanent_uuid: "e65182db1a254905939ae7ae864acfd7" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 34385 } }
I20260807 08:50:07.189821 26282 ts_tablet_manager.cc:1428] T 5656d45db93f4f3ab57832ad63ca399e P dccc17259c6e4d35bcf60aae6b6cdfe4: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260807 08:50:07.189863 26050 heartbeater.cc:503] Master 127.4.159.190:34273 was elected leader, sending a full tablet report...
I20260807 08:50:07.190176 26289 tablet_bootstrap.cc:492] T 5656d45db93f4f3ab57832ad63ca399e P e65182db1a254905939ae7ae864acfd7: No bootstrap required, opened a new log
I20260807 08:50:07.190253 26289 ts_tablet_manager.cc:1397] T 5656d45db93f4f3ab57832ad63ca399e P e65182db1a254905939ae7ae864acfd7: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 08:50:07.191742 26289 raft_consensus.cc:348] T 5656d45db93f4f3ab57832ad63ca399e P e65182db1a254905939ae7ae864acfd7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dccc17259c6e4d35bcf60aae6b6cdfe4" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 34071 } } peers { permanent_uuid: "9181e852d19c47d2bb93dec238a89ffb" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 46023 } } peers { permanent_uuid: "e65182db1a254905939ae7ae864acfd7" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 34385 } }
I20260807 08:50:07.191864 26289 raft_consensus.cc:374] T 5656d45db93f4f3ab57832ad63ca399e P e65182db1a254905939ae7ae864acfd7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:50:07.191884 26289 raft_consensus.cc:729] T 5656d45db93f4f3ab57832ad63ca399e P e65182db1a254905939ae7ae864acfd7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e65182db1a254905939ae7ae864acfd7, State: Initialized, Role: FOLLOWER
I20260807 08:50:07.191984 26289 consensus_queue.cc:260] T 5656d45db93f4f3ab57832ad63ca399e P e65182db1a254905939ae7ae864acfd7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dccc17259c6e4d35bcf60aae6b6cdfe4" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 34071 } } peers { permanent_uuid: "9181e852d19c47d2bb93dec238a89ffb" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 46023 } } peers { permanent_uuid: "e65182db1a254905939ae7ae864acfd7" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 34385 } }
I20260807 08:50:07.192217 26289 ts_tablet_manager.cc:1428] T 5656d45db93f4f3ab57832ad63ca399e P e65182db1a254905939ae7ae864acfd7: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260807 08:50:07.192235 26259 heartbeater.cc:503] Master 127.4.159.190:34273 was elected leader, sending a full tablet report...
I20260807 08:50:07.219378 26295 raft_consensus.cc:482] T 5656d45db93f4f3ab57832ad63ca399e P e65182db1a254905939ae7ae864acfd7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:50:07.219444 26295 raft_consensus.cc:504] T 5656d45db93f4f3ab57832ad63ca399e P e65182db1a254905939ae7ae864acfd7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dccc17259c6e4d35bcf60aae6b6cdfe4" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 34071 } } peers { permanent_uuid: "9181e852d19c47d2bb93dec238a89ffb" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 46023 } } peers { permanent_uuid: "e65182db1a254905939ae7ae864acfd7" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 34385 } }
I20260807 08:50:07.219699 26295 leader_election.cc:288] T 5656d45db93f4f3ab57832ad63ca399e P e65182db1a254905939ae7ae864acfd7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dccc17259c6e4d35bcf60aae6b6cdfe4 (127.4.159.130:34071), 9181e852d19c47d2bb93dec238a89ffb (127.4.159.129:46023)
I20260807 08:50:07.223181 26278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:07.219911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.131:49119 (local address 127.4.159.130:34071)
0807 08:50:07.220003 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0807 08:50:07.220005 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:50:07.220014 (+ 9us) server_negotiation.cc:407] Connection header received
0807 08:50:07.220039 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:50:07.220040 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:50:07.220049 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:50:07.220094 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:50:07.220250 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:07.220957 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:07.222276 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:07.222860 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:07.222931 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:50:07.222939 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0807 08:50:07.223086 (+ 147us) server_negotiation.cc:299] Negotiation successful
0807 08:50:07.223117 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 08:50:07.223181 26281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:07.219927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.131:42375 (local address 127.4.159.129:46023)
0807 08:50:07.219983 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0807 08:50:07.219985 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:50:07.220058 (+ 73us) server_negotiation.cc:407] Connection header received
0807 08:50:07.220152 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:50:07.220153 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:50:07.220180 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:50:07.220224 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:50:07.220357 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:07.221038 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:07.222275 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:07.222863 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:07.222932 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:50:07.222939 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0807 08:50:07.223086 (+ 147us) server_negotiation.cc:299] Negotiation successful
0807 08:50:07.223117 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 08:50:07.223220 26262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:07.219830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.130:34071 (local address 127.4.159.131:49119)
0807 08:50:07.219875 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:07.219886 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 08:50:07.219930 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:50:07.220100 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:50:07.220101 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:50:07.220103 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:50:07.220163 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:50:07.220164 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:50:07.221035 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:50:07.221036 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:50:07.222202 (+ 1166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:50:07.222205 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:50:07.222934 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:50:07.222935 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:50:07.223021 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:50:07.223022 (+ 1us) client_negotiation.cc:769] Sending connection context
0807 08:50:07.223076 (+ 54us) client_negotiation.cc:240] Negotiation successful
0807 08:50:07.223122 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27,"mutex_wait_us":8}
I20260807 08:50:07.223230 26297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:07.219830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.129:46023 (local address 127.4.159.131:42375)
0807 08:50:07.220000 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:07.220010 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 08:50:07.220068 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:50:07.220240 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:50:07.220241 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:50:07.220243 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:50:07.220308 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:50:07.220309 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:50:07.221100 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:50:07.221101 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:50:07.222203 (+ 1102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:50:07.222205 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:50:07.222934 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:50:07.222935 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:50:07.223018 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:50:07.223022 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 08:50:07.223076 (+ 54us) client_negotiation.cc:240] Negotiation successful
0807 08:50:07.223122 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"mutex_wait_us":50,"thread_start_us":83,"threads_started":1}
I20260807 08:50:07.223527 25987 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5656d45db93f4f3ab57832ad63ca399e" candidate_uuid: "e65182db1a254905939ae7ae864acfd7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dccc17259c6e4d35bcf60aae6b6cdfe4" is_pre_election: true
I20260807 08:50:07.223639 25987 raft_consensus.cc:2454] T 5656d45db93f4f3ab57832ad63ca399e P dccc17259c6e4d35bcf60aae6b6cdfe4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e65182db1a254905939ae7ae864acfd7 in term 0.
I20260807 08:50:07.223623 25777 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5656d45db93f4f3ab57832ad63ca399e" candidate_uuid: "e65182db1a254905939ae7ae864acfd7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9181e852d19c47d2bb93dec238a89ffb" is_pre_election: true
I20260807 08:50:07.223744 25777 raft_consensus.cc:2454] T 5656d45db93f4f3ab57832ad63ca399e P 9181e852d19c47d2bb93dec238a89ffb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e65182db1a254905939ae7ae864acfd7 in term 0.
I20260807 08:50:07.223860 26078 leader_election.cc:302] T 5656d45db93f4f3ab57832ad63ca399e P e65182db1a254905939ae7ae864acfd7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dccc17259c6e4d35bcf60aae6b6cdfe4, e65182db1a254905939ae7ae864acfd7; no voters:
I20260807 08:50:07.223971 26295 raft_consensus.cc:2781] T 5656d45db93f4f3ab57832ad63ca399e P e65182db1a254905939ae7ae864acfd7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:50:07.224028 26295 raft_consensus.cc:482] T 5656d45db93f4f3ab57832ad63ca399e P e65182db1a254905939ae7ae864acfd7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:50:07.224049 26295 raft_consensus.cc:3037] T 5656d45db93f4f3ab57832ad63ca399e P e65182db1a254905939ae7ae864acfd7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:50:07.224607 26295 raft_consensus.cc:504] T 5656d45db93f4f3ab57832ad63ca399e P e65182db1a254905939ae7ae864acfd7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dccc17259c6e4d35bcf60aae6b6cdfe4" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 34071 } } peers { permanent_uuid: "9181e852d19c47d2bb93dec238a89ffb" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 46023 } } peers { permanent_uuid: "e65182db1a254905939ae7ae864acfd7" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 34385 } }
I20260807 08:50:07.224715 26295 leader_election.cc:288] T 5656d45db93f4f3ab57832ad63ca399e P e65182db1a254905939ae7ae864acfd7 [CANDIDATE]: Term 1 election: Requested vote from peers dccc17259c6e4d35bcf60aae6b6cdfe4 (127.4.159.130:34071), 9181e852d19c47d2bb93dec238a89ffb (127.4.159.129:46023)
I20260807 08:50:07.224848 25777 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5656d45db93f4f3ab57832ad63ca399e" candidate_uuid: "e65182db1a254905939ae7ae864acfd7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9181e852d19c47d2bb93dec238a89ffb"
I20260807 08:50:07.224929 25777 raft_consensus.cc:3037] T 5656d45db93f4f3ab57832ad63ca399e P 9181e852d19c47d2bb93dec238a89ffb [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:50:07.224929 25987 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5656d45db93f4f3ab57832ad63ca399e" candidate_uuid: "e65182db1a254905939ae7ae864acfd7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dccc17259c6e4d35bcf60aae6b6cdfe4"
I20260807 08:50:07.224988 25987 raft_consensus.cc:3037] T 5656d45db93f4f3ab57832ad63ca399e P dccc17259c6e4d35bcf60aae6b6cdfe4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:50:07.225735 25987 raft_consensus.cc:2454] T 5656d45db93f4f3ab57832ad63ca399e P dccc17259c6e4d35bcf60aae6b6cdfe4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e65182db1a254905939ae7ae864acfd7 in term 1.
I20260807 08:50:07.225735 25777 raft_consensus.cc:2454] T 5656d45db93f4f3ab57832ad63ca399e P 9181e852d19c47d2bb93dec238a89ffb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e65182db1a254905939ae7ae864acfd7 in term 1.
I20260807 08:50:07.225885 26078 leader_election.cc:302] T 5656d45db93f4f3ab57832ad63ca399e P e65182db1a254905939ae7ae864acfd7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dccc17259c6e4d35bcf60aae6b6cdfe4, e65182db1a254905939ae7ae864acfd7; no voters:
I20260807 08:50:07.225977 26295 raft_consensus.cc:2781] T 5656d45db93f4f3ab57832ad63ca399e P e65182db1a254905939ae7ae864acfd7 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:50:07.226096 26295 raft_consensus.cc:686] T 5656d45db93f4f3ab57832ad63ca399e P e65182db1a254905939ae7ae864acfd7 [term 1 LEADER]: Becoming Leader. State: Replica: e65182db1a254905939ae7ae864acfd7, State: Running, Role: LEADER
I20260807 08:50:07.226169 26295 consensus_queue.cc:237] T 5656d45db93f4f3ab57832ad63ca399e P e65182db1a254905939ae7ae864acfd7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dccc17259c6e4d35bcf60aae6b6cdfe4" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 34071 } } peers { permanent_uuid: "9181e852d19c47d2bb93dec238a89ffb" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 46023 } } peers { permanent_uuid: "e65182db1a254905939ae7ae864acfd7" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 34385 } }
I20260807 08:50:07.226853 25564 catalog_manager.cc:5495] T 5656d45db93f4f3ab57832ad63ca399e P e65182db1a254905939ae7ae864acfd7 reported cstate change: term changed from 0 to 1, leader changed from <none> to e65182db1a254905939ae7ae864acfd7 (127.4.159.131). New cstate: current_term: 1 leader_uuid: "e65182db1a254905939ae7ae864acfd7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dccc17259c6e4d35bcf60aae6b6cdfe4" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 34071 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9181e852d19c47d2bb93dec238a89ffb" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 46023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e65182db1a254905939ae7ae864acfd7" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 34385 } health_report { overall_health: HEALTHY } } }
I20260807 08:50:07.235353 26279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:07.233541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40822 (local address 127.4.159.131:34385)
0807 08:50:07.233573 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0807 08:50:07.233575 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:50:07.233582 (+ 7us) server_negotiation.cc:407] Connection header received
0807 08:50:07.233607 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:50:07.233607 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:50:07.233619 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:50:07.233669 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:50:07.233760 (+ 91us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:07.234164 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:07.234824 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:07.234992 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:07.235035 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:50:07.235138 (+ 103us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:50:07.235213 (+ 75us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:50:07.235260 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0807 08:50:07.235271 (+ 11us) server_negotiation.cc:299] Negotiation successful
0807 08:50:07.235284 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260807 08:50:07.236726 25987 raft_consensus.cc:1261] T 5656d45db93f4f3ab57832ad63ca399e P dccc17259c6e4d35bcf60aae6b6cdfe4 [term 1 FOLLOWER]: Refusing update from remote peer e65182db1a254905939ae7ae864acfd7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:50:07.236726 25777 raft_consensus.cc:1261] T 5656d45db93f4f3ab57832ad63ca399e P 9181e852d19c47d2bb93dec238a89ffb [term 1 FOLLOWER]: Refusing update from remote peer e65182db1a254905939ae7ae864acfd7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:50:07.237037 26298 consensus_queue.cc:1035] T 5656d45db93f4f3ab57832ad63ca399e P e65182db1a254905939ae7ae864acfd7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9181e852d19c47d2bb93dec238a89ffb" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 46023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:50:07.237133 26295 consensus_queue.cc:1035] T 5656d45db93f4f3ab57832ad63ca399e P e65182db1a254905939ae7ae864acfd7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dccc17259c6e4d35bcf60aae6b6cdfe4" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 34071 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 08:50:07.238060 26051 tablet.cc:2369] T 5656d45db93f4f3ab57832ad63ca399e P dccc17259c6e4d35bcf60aae6b6cdfe4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 08:50:07.239804 26306 mvcc.cc:205] Tried to move back new op lower bound from 7315835319239688192 to 7315835319199092736. Current Snapshot: MvccSnapshot[applied={T|T < 7315835319239688192}]
I20260807 08:50:07.240321 26307 mvcc.cc:205] Tried to move back new op lower bound from 7315835319239688192 to 7315835319199092736. Current Snapshot: MvccSnapshot[applied={T|T < 7315835319239688192 or (T in {7315835319239688192})}]
I20260807 08:50:07.241389 25564 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59544:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:50:07.241464 25564 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59544:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:50:07.242056 25564 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7253adae706a42309f5b712541c5b57d: Sending DeleteTablet for 3 replicas of tablet 5656d45db93f4f3ab57832ad63ca399e
I20260807 08:50:07.242401 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 25658
I20260807 08:50:07.242393 25963 tablet_service.cc:1514] Processing DeleteTablet for tablet 5656d45db93f4f3ab57832ad63ca399e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:50:07 UTC) from {username='kudu', principal='kudu/127.4.159.190@KRBTEST.COM'} at 127.0.0.1:37402
I20260807 08:50:07.242424 26155 tablet_service.cc:1514] Processing DeleteTablet for tablet 5656d45db93f4f3ab57832ad63ca399e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:50:07 UTC) from {username='kudu', principal='kudu/127.4.159.190@KRBTEST.COM'} at 127.0.0.1:40812
I20260807 08:50:07.249096 26317 tablet_replica.cc:330] T 5656d45db93f4f3ab57832ad63ca399e P e65182db1a254905939ae7ae864acfd7: stopping tablet replica
I20260807 08:50:07.249183 26317 raft_consensus.cc:2229] T 5656d45db93f4f3ab57832ad63ca399e P e65182db1a254905939ae7ae864acfd7 [term 1 LEADER]: Raft consensus shutting down.
I20260807 08:50:07.249271 26317 raft_consensus.cc:2258] T 5656d45db93f4f3ab57832ad63ca399e P e65182db1a254905939ae7ae864acfd7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:50:07.251554 26317 ts_tablet_manager.cc:1905] T 5656d45db93f4f3ab57832ad63ca399e P e65182db1a254905939ae7ae864acfd7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:50:07.252646 26316 tablet_replica.cc:330] T 5656d45db93f4f3ab57832ad63ca399e P dccc17259c6e4d35bcf60aae6b6cdfe4: stopping tablet replica
I20260807 08:50:07.252739 26316 raft_consensus.cc:2229] T 5656d45db93f4f3ab57832ad63ca399e P dccc17259c6e4d35bcf60aae6b6cdfe4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:50:07.252836 26316 raft_consensus.cc:2258] T 5656d45db93f4f3ab57832ad63ca399e P dccc17259c6e4d35bcf60aae6b6cdfe4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:50:07.252993 26317 ts_tablet_manager.cc:1918] T 5656d45db93f4f3ab57832ad63ca399e P e65182db1a254905939ae7ae864acfd7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:50:07.253054 26317 log.cc:1196] T 5656d45db93f4f3ab57832ad63ca399e P e65182db1a254905939ae7ae864acfd7: Deleting WAL directory at /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-2/wal/wals/5656d45db93f4f3ab57832ad63ca399e
I20260807 08:50:07.253260 26317 ts_tablet_manager.cc:1939] T 5656d45db93f4f3ab57832ad63ca399e P e65182db1a254905939ae7ae864acfd7: Deleting consensus metadata
I20260807 08:50:07.253546 25539 catalog_manager.cc:4841] TS e65182db1a254905939ae7ae864acfd7 (127.4.159.131:34385): tablet 5656d45db93f4f3ab57832ad63ca399e (table test-table [id=22d6560a2be64420bba17c5a1de9ed50]) successfully deleted
I20260807 08:50:07.254725 26316 ts_tablet_manager.cc:1905] T 5656d45db93f4f3ab57832ad63ca399e P dccc17259c6e4d35bcf60aae6b6cdfe4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:50:07.256199 26316 ts_tablet_manager.cc:1918] T 5656d45db93f4f3ab57832ad63ca399e P dccc17259c6e4d35bcf60aae6b6cdfe4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:50:07.256280 26316 log.cc:1196] T 5656d45db93f4f3ab57832ad63ca399e P dccc17259c6e4d35bcf60aae6b6cdfe4: Deleting WAL directory at /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-1/wal/wals/5656d45db93f4f3ab57832ad63ca399e
I20260807 08:50:07.256496 26316 ts_tablet_manager.cc:1939] T 5656d45db93f4f3ab57832ad63ca399e P dccc17259c6e4d35bcf60aae6b6cdfe4: Deleting consensus metadata
I20260807 08:50:07.259989 25538 catalog_manager.cc:4841] TS dccc17259c6e4d35bcf60aae6b6cdfe4 (127.4.159.130:34071): tablet 5656d45db93f4f3ab57832ad63ca399e (table test-table [id=22d6560a2be64420bba17c5a1de9ed50]) successfully deleted
W20260807 08:50:07.265025 25542 connection.cc:535] client connection to 127.4.159.129:46023 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260807 08:50:07.265112 25542 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 08:50:07.265311 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 25862
I20260807 08:50:07.266789 26277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:07.265437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.129:46023 (local address 127.0.0.1:60252)
0807 08:50:07.266716 (+ 1279us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:07.266757 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.129:46023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1260}
W20260807 08:50:07.272186 25542 catalog_manager.cc:4584] TS 9181e852d19c47d2bb93dec238a89ffb (127.4.159.129:46023): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5656d45db93f4f3ab57832ad63ca399e: Network error: Client connection negotiation failed: client connection to 127.4.159.129:46023: connect: Connection refused (error 111)
I20260807 08:50:07.272279 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 26056
I20260807 08:50:07.287515 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 25514
I20260807 08:50:07.293879 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.159.190:34273
--webserver_interface=127.4.159.190
--webserver_port=46171
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab
--principal=kudu/127.4.159.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:35687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.159.190:34273
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:50:07.371943 26321 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:50:07.372115 26321 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:50:07.372136 26321 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:50:07.373210 26321 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:50:07.373255 26321 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:50:07.373268 26321 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:50:07.373291 26321 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:50:07.373314 26321 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:50:07.374737 26321 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:35687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.4.159.190:34273
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab
--principal=kudu/127.4.159.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.159.190:34273
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.159.190
--webserver_port=46171
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26321
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:50:07.374946 26321 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:50:07.375145 26321 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:50:07.383503 26331 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:50:07.383544 26334 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:50:07.383565 26321 server_base.cc:1034] running on GCE node
W20260807 08:50:07.383504 26337 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:50:07.383944 26321 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:50:07.384230 26321 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:50:07.385365 26321 hybrid_clock.cc:648] HybridClock initialized: now 1786092607385354 us; error 29 us; skew 500 ppm
Aug 07 08:50:07 dist-test-slave-6zbq krb5kdc[25496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092607, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:50:07.390014 26321 init.cc:377] Logged in from keytab as kudu/127.4.159.190@KRBTEST.COM (short username kudu)
I20260807 08:50:07.390439 26321 webserver.cc:460] Webserver started at http://127.4.159.190:46171/ using document root <none> and password file <none>
I20260807 08:50:07.390636 26321 fs_manager.cc:359] Metadata directory not provided
I20260807 08:50:07.390692 26321 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:50:07.391871 26321 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 08:50:07.392417 26350 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:50:07.392571 26321 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 08:50:07.392652 26321 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/master-0/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/master-0/wal
uuid: "7253adae706a42309f5b712541c5b57d"
format_stamp: "Formatted at 2026-08-07 08:50:06 on dist-test-slave-6zbq"
server_key: "a8bb7a8ab5a147d14a7be7e4363c27a0"
server_key_iv: "e8133f3cd3a115d8ba6b2db3505a635a"
server_key_version: "encryptionkey@0"
I20260807 08:50:07.392984 26321 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:50:07.409482 26321 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:50:07.411388 26321 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:50:07.411487 26321 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:50:07.415088 26321 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.190:34273
I20260807 08:50:07.415128 26427 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.190:34273 every 8 connection(s)
I20260807 08:50:07.415436 26321 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
I20260807 08:50:07.416944 26428 sys_catalog.cc:263] Verifying existing consensus state
I20260807 08:50:07.417570 26428 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7253adae706a42309f5b712541c5b57d: Bootstrap starting.
I20260807 08:50:07.418298 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 26321
I20260807 08:50:07.418604 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.159.129:46023
--local_ip_for_outbound_sockets=127.4.159.129
--tserver_master_addrs=127.4.159.190:34273
--webserver_port=41265
--webserver_interface=127.4.159.129
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.129.keytab
--principal=kudu/127.4.159.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:35687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 08:50:07.426367 26428 log.cc:824] T 00000000000000000000000000000000 P 7253adae706a42309f5b712541c5b57d: Log is configured to *not* fsync() on all Append() calls
I20260807 08:50:07.428037 26428 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7253adae706a42309f5b712541c5b57d: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 08:50:07.428282 26428 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7253adae706a42309f5b712541c5b57d: Bootstrap complete.
I20260807 08:50:07.430301 26428 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7253adae706a42309f5b712541c5b57d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7253adae706a42309f5b712541c5b57d" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 34273 } }
I20260807 08:50:07.430567 26428 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7253adae706a42309f5b712541c5b57d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7253adae706a42309f5b712541c5b57d, State: Initialized, Role: FOLLOWER
I20260807 08:50:07.430714 26428 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7253adae706a42309f5b712541c5b57d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7253adae706a42309f5b712541c5b57d" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 34273 } }
I20260807 08:50:07.430787 26428 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7253adae706a42309f5b712541c5b57d [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:50:07.430838 26428 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7253adae706a42309f5b712541c5b57d [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:50:07.430886 26428 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7253adae706a42309f5b712541c5b57d [term 1 FOLLOWER]: Advancing to term 2
I20260807 08:50:07.431802 26428 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7253adae706a42309f5b712541c5b57d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7253adae706a42309f5b712541c5b57d" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 34273 } }
I20260807 08:50:07.431895 26428 leader_election.cc:302] T 00000000000000000000000000000000 P 7253adae706a42309f5b712541c5b57d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7253adae706a42309f5b712541c5b57d; no voters:
I20260807 08:50:07.432036 26428 leader_election.cc:288] T 00000000000000000000000000000000 P 7253adae706a42309f5b712541c5b57d [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 08:50:07.432083 26432 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7253adae706a42309f5b712541c5b57d [term 2 FOLLOWER]: Leader election won for term 2
I20260807 08:50:07.432199 26432 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7253adae706a42309f5b712541c5b57d [term 2 LEADER]: Becoming Leader. State: Replica: 7253adae706a42309f5b712541c5b57d, State: Running, Role: LEADER
I20260807 08:50:07.432294 26428 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7253adae706a42309f5b712541c5b57d [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:50:07.432291 26432 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7253adae706a42309f5b712541c5b57d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7253adae706a42309f5b712541c5b57d" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 34273 } }
I20260807 08:50:07.432686 26433 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7253adae706a42309f5b712541c5b57d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "7253adae706a42309f5b712541c5b57d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7253adae706a42309f5b712541c5b57d" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 34273 } } }
I20260807 08:50:07.432770 26433 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7253adae706a42309f5b712541c5b57d [sys.catalog]: This master's current role is: LEADER
I20260807 08:50:07.432906 26432 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7253adae706a42309f5b712541c5b57d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7253adae706a42309f5b712541c5b57d. Latest consensus state: current_term: 2 leader_uuid: "7253adae706a42309f5b712541c5b57d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7253adae706a42309f5b712541c5b57d" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 34273 } } }
I20260807 08:50:07.432956 26432 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7253adae706a42309f5b712541c5b57d [sys.catalog]: This master's current role is: LEADER
I20260807 08:50:07.433076 26443 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:50:07.433830 26443 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:50:07.433929 26443 catalog_manager.cc:1259] Loaded cluster ID: e34fe26867a9489194587342451da3f2
I20260807 08:50:07.433959 26443 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:50:07.434271 26443 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260807 08:50:07.436940 26443 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260807 08:50:07.438635 26443 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:50:07.438805 26443 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 7253adae706a42309f5b712541c5b57d: Loaded TSK: 0
I20260807 08:50:07.438990 26443 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 08:50:07.514284 26430 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:50:07.514498 26430 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:50:07.514529 26430 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:50:07.515615 26430 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:50:07.515656 26430 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:50:07.515689 26430 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.159.129
I20260807 08:50:07.517076 26430 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:35687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.129.keytab
--principal=kudu/127.4.159.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.159.129:46023
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.159.129
--webserver_port=41265
--tserver_master_addrs=127.4.159.190:34273
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26430
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.159.129
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:50:07.517303 26430 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:50:07.517488 26430 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:50:07.521978 26476 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:50:07.522018 26472 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:50:07.522053 26474 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:50:07.522379 26430 server_base.cc:1034] running on GCE node
I20260807 08:50:07.522563 26430 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:50:07.522769 26430 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:50:07.523905 26430 hybrid_clock.cc:648] HybridClock initialized: now 1786092607523893 us; error 31 us; skew 500 ppm
Aug 07 08:50:07 dist-test-slave-6zbq krb5kdc[25496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092607, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:50:07.526194 26430 init.cc:377] Logged in from keytab as kudu/127.4.159.129@KRBTEST.COM (short username kudu)
I20260807 08:50:07.526629 26430 webserver.cc:460] Webserver started at http://127.4.159.129:41265/ using document root <none> and password file <none>
I20260807 08:50:07.526823 26430 fs_manager.cc:359] Metadata directory not provided
I20260807 08:50:07.526875 26430 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:50:07.528105 26430 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260807 08:50:07.528697 26485 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:50:07.528848 26430 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 08:50:07.528929 26430 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-0/wal
uuid: "9181e852d19c47d2bb93dec238a89ffb"
format_stamp: "Formatted at 2026-08-07 08:50:06 on dist-test-slave-6zbq"
server_key: "f0e514983f7d1a8dc026d8457e216f4a"
server_key_iv: "7858241c60bdeb4e4b048a12f6a3c95b"
server_key_version: "encryptionkey@0"
I20260807 08:50:07.529219 26430 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:50:07.537629 26430 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:50:07.539438 26430 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:50:07.539542 26430 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:50:07.539739 26430 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:50:07.540170 26496 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260807 08:50:07.541437 26430 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 08:50:07.541486 26430 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.001s user 0.000s sys 0.000s
I20260807 08:50:07.541507 26430 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 08:50:07.542068 26430 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 08:50:07.542119 26430 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260807 08:50:07.542171 26496 tablet_bootstrap.cc:492] T 5656d45db93f4f3ab57832ad63ca399e P 9181e852d19c47d2bb93dec238a89ffb: Bootstrap starting.
I20260807 08:50:07.549402 26430 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.129:46023
I20260807 08:50:07.549855 26430 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-0/data/info.pb
I20260807 08:50:07.550267 26650 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.129:46023 every 8 connection(s)
I20260807 08:50:07.552843 26496 log.cc:824] T 5656d45db93f4f3ab57832ad63ca399e P 9181e852d19c47d2bb93dec238a89ffb: Log is configured to *not* fsync() on all Append() calls
I20260807 08:50:07.553869 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 26430
I20260807 08:50:07.554138 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.159.130:34071
--local_ip_for_outbound_sockets=127.4.159.130
--tserver_master_addrs=127.4.159.190:34273
--webserver_port=38387
--webserver_interface=127.4.159.130
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.130.keytab
--principal=kudu/127.4.159.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:35687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 08:50:07.555114 26496 tablet_bootstrap.cc:492] T 5656d45db93f4f3ab57832ad63ca399e P 9181e852d19c47d2bb93dec238a89ffb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 08:50:07.555356 26496 tablet_bootstrap.cc:492] T 5656d45db93f4f3ab57832ad63ca399e P 9181e852d19c47d2bb93dec238a89ffb: Bootstrap complete.
I20260807 08:50:07.556394 26496 ts_tablet_manager.cc:1397] T 5656d45db93f4f3ab57832ad63ca399e P 9181e852d19c47d2bb93dec238a89ffb: Time spent bootstrapping tablet: real 0.014s user 0.009s sys 0.004s
I20260807 08:50:07.557631 26496 raft_consensus.cc:348] T 5656d45db93f4f3ab57832ad63ca399e P 9181e852d19c47d2bb93dec238a89ffb [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dccc17259c6e4d35bcf60aae6b6cdfe4" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 34071 } } peers { permanent_uuid: "9181e852d19c47d2bb93dec238a89ffb" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 46023 } } peers { permanent_uuid: "e65182db1a254905939ae7ae864acfd7" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 34385 } }
I20260807 08:50:07.557854 26496 raft_consensus.cc:729] T 5656d45db93f4f3ab57832ad63ca399e P 9181e852d19c47d2bb93dec238a89ffb [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9181e852d19c47d2bb93dec238a89ffb, State: Initialized, Role: FOLLOWER
I20260807 08:50:07.557994 26496 consensus_queue.cc:260] T 5656d45db93f4f3ab57832ad63ca399e P 9181e852d19c47d2bb93dec238a89ffb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dccc17259c6e4d35bcf60aae6b6cdfe4" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 34071 } } peers { permanent_uuid: "9181e852d19c47d2bb93dec238a89ffb" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 46023 } } peers { permanent_uuid: "e65182db1a254905939ae7ae864acfd7" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 34385 } }
I20260807 08:50:07.558275 26496 ts_tablet_manager.cc:1428] T 5656d45db93f4f3ab57832ad63ca399e P 9181e852d19c47d2bb93dec238a89ffb: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
Aug 07 08:50:07 dist-test-slave-6zbq krb5kdc[25496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092607, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.129@KRBTEST.COM for kudu/127.4.159.190@KRBTEST.COM
I20260807 08:50:07.563162 26655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:07.550557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:34273 (local address 127.4.159.129:37097)
0807 08:50:07.550896 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:07.550938 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 08:50:07.551451 (+ 513us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:50:07.552114 (+ 663us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:50:07.552121 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:50:07.552488 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:50:07.552757 (+ 269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:50:07.552763 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:50:07.554496 (+ 1733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:50:07.554497 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:50:07.555759 (+ 1262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:50:07.555762 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:50:07.556596 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:50:07.556597 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:50:07.556751 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:50:07.557298 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:50:07.557317 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:50:07.559180 (+ 1863us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:50:07.561362 (+ 2182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:50:07.561369 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:50:07.561371 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:50:07.561637 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:50:07.561901 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:50:07.561902 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:50:07.561902 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:50:07.562049 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:50:07.562493 (+ 444us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:50:07.562500 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:50:07.562659 (+ 159us) client_negotiation.cc:769] Sending connection context
0807 08:50:07.562781 (+ 122us) client_negotiation.cc:240] Negotiation successful
0807 08:50:07.562950 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":85,"threads_started":1}
I20260807 08:50:07.563174 26657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:07.550922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.129:37097 (local address 127.4.159.190:34273)
0807 08:50:07.551637 (+ 715us) server_negotiation.cc:206] Beginning negotiation
0807 08:50:07.551646 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 08:50:07.551680 (+ 34us) server_negotiation.cc:407] Connection header received
0807 08:50:07.551817 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:50:07.551823 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:50:07.551877 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:50:07.552004 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:50:07.552905 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:07.554334 (+ 1429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:07.555875 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:07.556471 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:07.557459 (+ 988us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:50:07.559298 (+ 1839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:50:07.559302 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:50:07.559302 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:50:07.559314 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:50:07.561267 (+ 1953us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:50:07.561726 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:50:07.561728 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:50:07.561729 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:50:07.561829 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:50:07.562130 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:50:07.562131 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:50:07.562131 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:50:07.562422 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:50:07.562500 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 08:50:07.562836 (+ 336us) server_negotiation.cc:299] Negotiation successful
0807 08:50:07.562968 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":669,"thread_start_us":82,"threads_started":1}
I20260807 08:50:07.564254 26652 heartbeater.cc:347] Connected to a master server at 127.4.159.190:34273
I20260807 08:50:07.564339 26652 heartbeater.cc:464] Registering TS with master...
I20260807 08:50:07.564492 26652 heartbeater.cc:511] Master 127.4.159.190:34273 requested a full tablet report, sending...
I20260807 08:50:07.565227 26376 ts_manager.cc:194] Registered new tserver with Master: 9181e852d19c47d2bb93dec238a89ffb (127.4.159.129:46023)
I20260807 08:50:07.566421 26376 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.159.129@KRBTEST.COM'} at 127.4.159.129:37097
I20260807 08:50:07.567530 26652 heartbeater.cc:503] Master 127.4.159.190:34273 was elected leader, sending a full tablet report...
I20260807 08:50:07.571933 26663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:07.565815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.129:46023 (local address 127.0.0.1:60256)
0807 08:50:07.568068 (+ 2253us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:07.568100 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0807 08:50:07.568177 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:50:07.568365 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:50:07.568367 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:50:07.568369 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:50:07.568423 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:50:07.568424 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:50:07.569618 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:50:07.569619 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:50:07.570810 (+ 1191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:50:07.570814 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:50:07.571674 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:50:07.571674 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:50:07.571748 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:50:07.571760 (+ 12us) client_negotiation.cc:769] Sending connection context
0807 08:50:07.571816 (+ 56us) client_negotiation.cc:240] Negotiation successful
0807 08:50:07.571860 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2231,"thread_start_us":87,"threads_started":1}
I20260807 08:50:07.571945 26665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:07.566726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60256 (local address 127.4.159.129:46023)
0807 08:50:07.566877 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0807 08:50:07.566882 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:50:07.568164 (+ 1282us) server_negotiation.cc:407] Connection header received
0807 08:50:07.568257 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:50:07.568259 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:50:07.568274 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:50:07.568355 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:50:07.568480 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:07.569533 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:07.570886 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:07.571589 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:07.571661 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:50:07.571720 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 08:50:07.571840 (+ 120us) server_negotiation.cc:299] Negotiation successful
0807 08:50:07.571880 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":78,"threads_started":1}
I20260807 08:50:07.572623 26527 tablet_service.cc:1514] Processing DeleteTablet for tablet 5656d45db93f4f3ab57832ad63ca399e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:50:07 UTC) from {username='kudu', principal='kudu/127.4.159.190@KRBTEST.COM'} at 127.0.0.1:60256
I20260807 08:50:07.573143 26666 tablet_replica.cc:330] stopping tablet replica
I20260807 08:50:07.573248 26666 raft_consensus.cc:2229] T 5656d45db93f4f3ab57832ad63ca399e P 9181e852d19c47d2bb93dec238a89ffb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:50:07.573305 26666 raft_consensus.cc:2258] T 5656d45db93f4f3ab57832ad63ca399e P 9181e852d19c47d2bb93dec238a89ffb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:50:07.573565 26666 ts_tablet_manager.cc:1905] T 5656d45db93f4f3ab57832ad63ca399e P 9181e852d19c47d2bb93dec238a89ffb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:50:07.575637 26666 ts_tablet_manager.cc:1918] T 5656d45db93f4f3ab57832ad63ca399e P 9181e852d19c47d2bb93dec238a89ffb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:50:07.575711 26666 log.cc:1196] T 5656d45db93f4f3ab57832ad63ca399e P 9181e852d19c47d2bb93dec238a89ffb: Deleting WAL directory at /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-0/wal/wals/5656d45db93f4f3ab57832ad63ca399e
I20260807 08:50:07.575903 26666 ts_tablet_manager.cc:1939] T 5656d45db93f4f3ab57832ad63ca399e P 9181e852d19c47d2bb93dec238a89ffb: Deleting consensus metadata
I20260807 08:50:07.576364 26355 catalog_manager.cc:4841] TS 9181e852d19c47d2bb93dec238a89ffb (127.4.159.129:46023): tablet 5656d45db93f4f3ab57832ad63ca399e (table test-table [id=22d6560a2be64420bba17c5a1de9ed50]) successfully deleted
W20260807 08:50:07.639410 26660 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:50:07.639626 26660 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:50:07.639659 26660 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:50:07.640794 26660 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:50:07.640833 26660 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:50:07.640867 26660 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.159.130
I20260807 08:50:07.642411 26660 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:35687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.130.keytab
--principal=kudu/127.4.159.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.159.130:34071
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.159.130
--webserver_port=38387
--tserver_master_addrs=127.4.159.190:34273
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26660
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.159.130
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:50:07.642663 26660 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:50:07.642884 26660 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:50:07.647888 26677 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:50:07.647943 26673 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:50:07.647889 26674 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:50:07.648010 26660 server_base.cc:1034] running on GCE node
I20260807 08:50:07.648269 26660 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:50:07.648463 26660 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:50:07.649596 26660 hybrid_clock.cc:648] HybridClock initialized: now 1786092607649579 us; error 29 us; skew 500 ppm
Aug 07 08:50:07 dist-test-slave-6zbq krb5kdc[25496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092607, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:50:07.651911 26660 init.cc:377] Logged in from keytab as kudu/127.4.159.130@KRBTEST.COM (short username kudu)
I20260807 08:50:07.652364 26660 webserver.cc:460] Webserver started at http://127.4.159.130:38387/ using document root <none> and password file <none>
I20260807 08:50:07.652582 26660 fs_manager.cc:359] Metadata directory not provided
I20260807 08:50:07.652720 26660 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:50:07.654032 26660 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:50:07.654647 26687 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:50:07.654801 26660 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 08:50:07.654888 26660 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-1/wal
uuid: "dccc17259c6e4d35bcf60aae6b6cdfe4"
format_stamp: "Formatted at 2026-08-07 08:50:06 on dist-test-slave-6zbq"
server_key: "f0d427e8337b42861d2f308790acd1f1"
server_key_iv: "d4f7e16f06cac3bc5945628c3ad319eb"
server_key_version: "encryptionkey@0"
I20260807 08:50:07.655234 26660 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:50:07.663409 26660 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:50:07.665380 26660 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:50:07.665503 26660 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:50:07.665688 26660 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:50:07.665957 26660 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:50:07.665989 26660 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:50:07.666015 26660 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:50:07.666041 26660 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:50:07.671474 26660 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.130:34071
I20260807 08:50:07.671535 26857 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.130:34071 every 8 connection(s)
I20260807 08:50:07.671825 26660 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-1/data/info.pb
Aug 07 08:50:07 dist-test-slave-6zbq krb5kdc[25496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092607, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.130@KRBTEST.COM for kudu/127.4.159.190@KRBTEST.COM
I20260807 08:50:07.679947 26657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:07.672470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.130:33197 (local address 127.4.159.190:34273)
0807 08:50:07.672525 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0807 08:50:07.672527 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:50:07.672721 (+ 194us) server_negotiation.cc:407] Connection header received
0807 08:50:07.673189 (+ 468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:50:07.673191 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:50:07.673206 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:50:07.673258 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:50:07.674110 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:07.674653 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:07.675584 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:07.675794 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:07.675849 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:50:07.677631 (+ 1782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:50:07.677633 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:50:07.677633 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:50:07.677641 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:50:07.678759 (+ 1118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:50:07.679100 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:50:07.679101 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:50:07.679102 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:50:07.679154 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:50:07.679364 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:50:07.679365 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:50:07.679365 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:50:07.679468 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:50:07.679523 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 08:50:07.679769 (+ 246us) server_negotiation.cc:299] Negotiation successful
0807 08:50:07.679856 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260807 08:50:07.680070 26863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:07.672443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:34273 (local address 127.4.159.130:33197)
0807 08:50:07.672604 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:07.672637 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0807 08:50:07.673098 (+ 461us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:50:07.673323 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:50:07.673327 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:50:07.673843 (+ 516us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:50:07.674015 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:50:07.674020 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:50:07.674742 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:50:07.674743 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:50:07.675503 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:50:07.675505 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:50:07.675863 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:50:07.675864 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:50:07.675976 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:50:07.676336 (+ 360us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:50:07.676344 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:50:07.677543 (+ 1199us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:50:07.678828 (+ 1285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:50:07.678834 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:50:07.678836 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:50:07.679032 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:50:07.679208 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:50:07.679209 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:50:07.679209 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:50:07.679310 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:50:07.679523 (+ 213us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:50:07.679530 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:50:07.679652 (+ 122us) client_negotiation.cc:769] Sending connection context
0807 08:50:07.679734 (+ 82us) client_negotiation.cc:240] Negotiation successful
0807 08:50:07.679875 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":88,"threads_started":1}
I20260807 08:50:07.680510 26860 heartbeater.cc:347] Connected to a master server at 127.4.159.190:34273
I20260807 08:50:07.680593 26860 heartbeater.cc:464] Registering TS with master...
I20260807 08:50:07.680729 26860 heartbeater.cc:511] Master 127.4.159.190:34273 requested a full tablet report, sending...
I20260807 08:50:07.681057 26376 ts_manager.cc:194] Registered new tserver with Master: dccc17259c6e4d35bcf60aae6b6cdfe4 (127.4.159.130:34071)
I20260807 08:50:07.681420 26376 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.159.130@KRBTEST.COM'} at 127.4.159.130:33197
I20260807 08:50:07.681586 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 26660
I20260807 08:50:07.681838 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.159.131:34385
--local_ip_for_outbound_sockets=127.4.159.131
--tserver_master_addrs=127.4.159.190:34273
--webserver_port=45051
--webserver_interface=127.4.159.131
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.131.keytab
--principal=kudu/127.4.159.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:35687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:50:07.760743 26864 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:50:07.760917 26864 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:50:07.760939 26864 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:50:07.762048 26864 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:50:07.762087 26864 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:50:07.762112 26864 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.159.131
I20260807 08:50:07.763497 26864 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:35687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.131.keytab
--principal=kudu/127.4.159.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.159.131:34385
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.159.131
--webserver_port=45051
--tserver_master_addrs=127.4.159.190:34273
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26864
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.159.131
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:50:07.763666 26864 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:50:07.763831 26864 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:50:07.768352 26876 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:50:07.768388 26864 server_base.cc:1034] running on GCE node
W20260807 08:50:07.768361 26881 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:50:07.768361 26884 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:50:07.768666 26864 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:50:07.768884 26864 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:50:07.770030 26864 hybrid_clock.cc:648] HybridClock initialized: now 1786092607770013 us; error 29 us; skew 500 ppm
Aug 07 08:50:07 dist-test-slave-6zbq krb5kdc[25496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092607, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:50:07.772356 26864 init.cc:377] Logged in from keytab as kudu/127.4.159.131@KRBTEST.COM (short username kudu)
I20260807 08:50:07.772732 26864 webserver.cc:460] Webserver started at http://127.4.159.131:45051/ using document root <none> and password file <none>
I20260807 08:50:07.772929 26864 fs_manager.cc:359] Metadata directory not provided
I20260807 08:50:07.772989 26864 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:50:07.774211 26864 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:50:07.774786 26893 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:50:07.774923 26864 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 08:50:07.775002 26864 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-2/wal
uuid: "e65182db1a254905939ae7ae864acfd7"
format_stamp: "Formatted at 2026-08-07 08:50:07 on dist-test-slave-6zbq"
server_key: "9ef9a62a6ed0d08b081ae4530045c96c"
server_key_iv: "db5536153bf901ec0f6dc22acf5ad28e"
server_key_version: "encryptionkey@0"
I20260807 08:50:07.775278 26864 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:50:07.788177 26864 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:50:07.790059 26864 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:50:07.790182 26864 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:50:07.790390 26864 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:50:07.790656 26864 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:50:07.790688 26864 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:50:07.790714 26864 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:50:07.790737 26864 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:50:07.796111 26864 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.131:34385
I20260807 08:50:07.796161 27055 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.131:34385 every 8 connection(s)
I20260807 08:50:07.796458 26864 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-2/data/info.pb
I20260807 08:50:07.796823 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 26864
Aug 07 08:50:07 dist-test-slave-6zbq krb5kdc[25496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092607, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.131@KRBTEST.COM for kudu/127.4.159.190@KRBTEST.COM
I20260807 08:50:07.803258 27061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:07.797211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59552 (local address 127.4.159.190:34273)
0807 08:50:07.797310 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0807 08:50:07.797312 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:50:07.797320 (+ 8us) server_negotiation.cc:407] Connection header received
0807 08:50:07.797401 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:50:07.797402 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:50:07.797414 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:50:07.797470 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:50:07.798000 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:07.798572 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:07.799298 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:07.799494 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:07.799548 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:50:07.800671 (+ 1123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:50:07.800672 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:50:07.800673 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:50:07.800679 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:50:07.802103 (+ 1424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:50:07.802499 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:50:07.802500 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:50:07.802501 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:50:07.802570 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:50:07.802757 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:50:07.802757 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:50:07.802758 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:50:07.802859 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:50:07.803020 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0807 08:50:07.803083 (+ 63us) server_negotiation.cc:299] Negotiation successful
0807 08:50:07.803165 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85,"thread_start_us":43,"threads_started":1}
I20260807 08:50:07.804980 26657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:07.797088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.131:60023 (local address 127.4.159.190:34273)
0807 08:50:07.797149 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0807 08:50:07.797152 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:50:07.797346 (+ 194us) server_negotiation.cc:407] Connection header received
0807 08:50:07.797795 (+ 449us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:50:07.797796 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:50:07.797808 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:50:07.797859 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:50:07.798747 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:07.799218 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:07.800250 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:07.800446 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:07.800499 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:50:07.802683 (+ 2184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:50:07.802684 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:50:07.802685 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:50:07.802691 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:50:07.804024 (+ 1333us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:50:07.804342 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:50:07.804342 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:50:07.804342 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:50:07.804384 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:50:07.804550 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:50:07.804551 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:50:07.804551 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:50:07.804633 (+ 82us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:50:07.804673 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0807 08:50:07.804871 (+ 198us) server_negotiation.cc:299] Negotiation successful
0807 08:50:07.804922 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260807 08:50:07.805102 27058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:07.797071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:34273 (local address 127.4.159.131:60023)
0807 08:50:07.797218 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:07.797256 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 08:50:07.797711 (+ 455us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:50:07.797928 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:50:07.797932 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:50:07.798490 (+ 558us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:50:07.798664 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:50:07.798669 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:50:07.799296 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:50:07.799297 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:50:07.800174 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:50:07.800177 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:50:07.800512 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:50:07.800513 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:50:07.800622 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:50:07.801018 (+ 396us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:50:07.801029 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:50:07.802592 (+ 1563us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:50:07.804080 (+ 1488us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:50:07.804085 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:50:07.804087 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:50:07.804286 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:50:07.804427 (+ 141us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:50:07.804427 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:50:07.804427 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:50:07.804511 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:50:07.804672 (+ 161us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:50:07.804675 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:50:07.804781 (+ 106us) client_negotiation.cc:769] Sending connection context
0807 08:50:07.804853 (+ 72us) client_negotiation.cc:240] Negotiation successful
0807 08:50:07.804960 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":82,"threads_started":1}
I20260807 08:50:07.805454 27056 heartbeater.cc:347] Connected to a master server at 127.4.159.190:34273
I20260807 08:50:07.805532 27056 heartbeater.cc:464] Registering TS with master...
I20260807 08:50:07.805689 27056 heartbeater.cc:511] Master 127.4.159.190:34273 requested a full tablet report, sending...
I20260807 08:50:07.806013 26376 ts_manager.cc:194] Registered new tserver with Master: e65182db1a254905939ae7ae864acfd7 (127.4.159.131:34385)
I20260807 08:50:07.806391 26376 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.159.131@KRBTEST.COM'} at 127.4.159.131:60023
I20260807 08:50:07.814558 4734 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 08:50:07.816988 26657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:07.814845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59556 (local address 127.4.159.190:34273)
0807 08:50:07.814880 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0807 08:50:07.814881 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 08:50:07.814984 (+ 103us) server_negotiation.cc:407] Connection header received
0807 08:50:07.815055 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:50:07.815055 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:50:07.815090 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:50:07.815144 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:50:07.815239 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:07.815667 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:07.816366 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:07.816541 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:07.816592 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:50:07.816732 (+ 140us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:50:07.816821 (+ 89us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:50:07.816887 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 08:50:07.816910 (+ 23us) server_negotiation.cc:299] Negotiation successful
0807 08:50:07.816926 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 08:50:07.817224 26376 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:59556:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 08:50:07.818813 26376 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:50:07.820755 26527 tablet_service.cc:1467] Processing CreateTablet for tablet b8b2b38f85eb452eb7b064fb2875339b (DEFAULT_TABLE table=test-table [id=4511ad8077ba4bdeb1be82ce4aabff6b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:50:07.820946 26527 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b8b2b38f85eb452eb7b064fb2875339b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:50:07.822301 26496 tablet_bootstrap.cc:492] T b8b2b38f85eb452eb7b064fb2875339b P 9181e852d19c47d2bb93dec238a89ffb: Bootstrap starting.
I20260807 08:50:07.822635 26496 tablet_bootstrap.cc:654] T b8b2b38f85eb452eb7b064fb2875339b P 9181e852d19c47d2bb93dec238a89ffb: Neither blocks nor log segments found. Creating new log.
I20260807 08:50:07.823199 26496 tablet_bootstrap.cc:492] T b8b2b38f85eb452eb7b064fb2875339b P 9181e852d19c47d2bb93dec238a89ffb: No bootstrap required, opened a new log
I20260807 08:50:07.823249 26496 ts_tablet_manager.cc:1397] T b8b2b38f85eb452eb7b064fb2875339b P 9181e852d19c47d2bb93dec238a89ffb: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260807 08:50:07.823375 26496 raft_consensus.cc:348] T b8b2b38f85eb452eb7b064fb2875339b P 9181e852d19c47d2bb93dec238a89ffb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dccc17259c6e4d35bcf60aae6b6cdfe4" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 34071 } } peers { permanent_uuid: "e65182db1a254905939ae7ae864acfd7" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 34385 } } peers { permanent_uuid: "9181e852d19c47d2bb93dec238a89ffb" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 46023 } }
I20260807 08:50:07.823448 26496 raft_consensus.cc:374] T b8b2b38f85eb452eb7b064fb2875339b P 9181e852d19c47d2bb93dec238a89ffb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:50:07.823500 26496 raft_consensus.cc:729] T b8b2b38f85eb452eb7b064fb2875339b P 9181e852d19c47d2bb93dec238a89ffb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9181e852d19c47d2bb93dec238a89ffb, State: Initialized, Role: FOLLOWER
I20260807 08:50:07.823542 26496 consensus_queue.cc:260] T b8b2b38f85eb452eb7b064fb2875339b P 9181e852d19c47d2bb93dec238a89ffb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dccc17259c6e4d35bcf60aae6b6cdfe4" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 34071 } } peers { permanent_uuid: "e65182db1a254905939ae7ae864acfd7" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 34385 } } peers { permanent_uuid: "9181e852d19c47d2bb93dec238a89ffb" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 46023 } }
I20260807 08:50:07.823628 26496 ts_tablet_manager.cc:1428] T b8b2b38f85eb452eb7b064fb2875339b P 9181e852d19c47d2bb93dec238a89ffb: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260807 08:50:07.824344 26663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:07.820494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.130:34071 (local address 127.0.0.1:37414)
0807 08:50:07.820538 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:07.820548 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 08:50:07.820585 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:50:07.820981 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:50:07.820982 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:50:07.820984 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:50:07.821019 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:50:07.821020 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:50:07.822318 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:50:07.822319 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:50:07.823225 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:50:07.823228 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:50:07.824146 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:50:07.824147 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:50:07.824203 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:50:07.824206 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 08:50:07.824232 (+ 26us) client_negotiation.cc:240] Negotiation successful
0807 08:50:07.824272 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260807 08:50:07.824414 27065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:07.820810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40824 (local address 127.4.159.131:34385)
0807 08:50:07.820956 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0807 08:50:07.820960 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:50:07.820968 (+ 8us) server_negotiation.cc:407] Connection header received
0807 08:50:07.820997 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:50:07.820998 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:50:07.821010 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:50:07.821220 (+ 210us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:50:07.821270 (+ 50us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:07.822269 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:07.823392 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:07.824050 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:07.824116 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:50:07.824167 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0807 08:50:07.824325 (+ 158us) server_negotiation.cc:299] Negotiation successful
0807 08:50:07.824356 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":69,"threads_started":1}
I20260807 08:50:07.824422 27063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:07.820499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.131:34385 (local address 127.0.0.1:40824)
0807 08:50:07.820677 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:07.820685 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0807 08:50:07.820720 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:50:07.821116 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:50:07.821117 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:50:07.821120 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:50:07.821155 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:50:07.821156 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:50:07.822366 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:50:07.822367 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:50:07.823313 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:50:07.823318 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:50:07.824195 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:50:07.824196 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:50:07.824247 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:50:07.824249 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 08:50:07.824299 (+ 50us) client_negotiation.cc:240] Negotiation successful
0807 08:50:07.824333 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"mutex_wait_us":11,"thread_start_us":86,"threads_started":1}
I20260807 08:50:07.824705 27064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:07.820649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37414 (local address 127.4.159.130:34071)
0807 08:50:07.820822 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0807 08:50:07.820827 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:50:07.820838 (+ 11us) server_negotiation.cc:407] Connection header received
0807 08:50:07.820878 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:50:07.820881 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:50:07.820897 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:50:07.820980 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:50:07.821091 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:07.822190 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:07.823335 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:07.824053 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:07.824515 (+ 462us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:50:07.824582 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 08:50:07.824606 (+ 24us) server_negotiation.cc:299] Negotiation successful
0807 08:50:07.824643 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":88,"threads_started":1}
I20260807 08:50:07.825084 26963 tablet_service.cc:1467] Processing CreateTablet for tablet b8b2b38f85eb452eb7b064fb2875339b (DEFAULT_TABLE table=test-table [id=4511ad8077ba4bdeb1be82ce4aabff6b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:50:07.825407 26963 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b8b2b38f85eb452eb7b064fb2875339b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:50:07.825335 26752 tablet_service.cc:1467] Processing CreateTablet for tablet b8b2b38f85eb452eb7b064fb2875339b (DEFAULT_TABLE table=test-table [id=4511ad8077ba4bdeb1be82ce4aabff6b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:50:07.825630 26752 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b8b2b38f85eb452eb7b064fb2875339b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:50:07.828707 27070 tablet_bootstrap.cc:492] T b8b2b38f85eb452eb7b064fb2875339b P e65182db1a254905939ae7ae864acfd7: Bootstrap starting.
I20260807 08:50:07.828917 27071 tablet_bootstrap.cc:492] T b8b2b38f85eb452eb7b064fb2875339b P dccc17259c6e4d35bcf60aae6b6cdfe4: Bootstrap starting.
I20260807 08:50:07.829466 27071 tablet_bootstrap.cc:654] T b8b2b38f85eb452eb7b064fb2875339b P dccc17259c6e4d35bcf60aae6b6cdfe4: Neither blocks nor log segments found. Creating new log.
I20260807 08:50:07.829897 27071 log.cc:824] T b8b2b38f85eb452eb7b064fb2875339b P dccc17259c6e4d35bcf60aae6b6cdfe4: Log is configured to *not* fsync() on all Append() calls
I20260807 08:50:07.830550 27070 tablet_bootstrap.cc:654] T b8b2b38f85eb452eb7b064fb2875339b P e65182db1a254905939ae7ae864acfd7: Neither blocks nor log segments found. Creating new log.
I20260807 08:50:07.830564 27071 tablet_bootstrap.cc:492] T b8b2b38f85eb452eb7b064fb2875339b P dccc17259c6e4d35bcf60aae6b6cdfe4: No bootstrap required, opened a new log
I20260807 08:50:07.830608 27071 ts_tablet_manager.cc:1397] T b8b2b38f85eb452eb7b064fb2875339b P dccc17259c6e4d35bcf60aae6b6cdfe4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 08:50:07.830977 27070 log.cc:824] T b8b2b38f85eb452eb7b064fb2875339b P e65182db1a254905939ae7ae864acfd7: Log is configured to *not* fsync() on all Append() calls
I20260807 08:50:07.831578 27070 tablet_bootstrap.cc:492] T b8b2b38f85eb452eb7b064fb2875339b P e65182db1a254905939ae7ae864acfd7: No bootstrap required, opened a new log
I20260807 08:50:07.831635 27070 ts_tablet_manager.cc:1397] T b8b2b38f85eb452eb7b064fb2875339b P e65182db1a254905939ae7ae864acfd7: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260807 08:50:07.832011 27071 raft_consensus.cc:348] T b8b2b38f85eb452eb7b064fb2875339b P dccc17259c6e4d35bcf60aae6b6cdfe4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dccc17259c6e4d35bcf60aae6b6cdfe4" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 34071 } } peers { permanent_uuid: "e65182db1a254905939ae7ae864acfd7" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 34385 } } peers { permanent_uuid: "9181e852d19c47d2bb93dec238a89ffb" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 46023 } }
I20260807 08:50:07.832137 27071 raft_consensus.cc:374] T b8b2b38f85eb452eb7b064fb2875339b P dccc17259c6e4d35bcf60aae6b6cdfe4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:50:07.832167 27071 raft_consensus.cc:729] T b8b2b38f85eb452eb7b064fb2875339b P dccc17259c6e4d35bcf60aae6b6cdfe4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dccc17259c6e4d35bcf60aae6b6cdfe4, State: Initialized, Role: FOLLOWER
I20260807 08:50:07.832276 27071 consensus_queue.cc:260] T b8b2b38f85eb452eb7b064fb2875339b P dccc17259c6e4d35bcf60aae6b6cdfe4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dccc17259c6e4d35bcf60aae6b6cdfe4" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 34071 } } peers { permanent_uuid: "e65182db1a254905939ae7ae864acfd7" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 34385 } } peers { permanent_uuid: "9181e852d19c47d2bb93dec238a89ffb" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 46023 } }
I20260807 08:50:07.832548 26860 heartbeater.cc:503] Master 127.4.159.190:34273 was elected leader, sending a full tablet report...
I20260807 08:50:07.832839 27071 ts_tablet_manager.cc:1428] T b8b2b38f85eb452eb7b064fb2875339b P dccc17259c6e4d35bcf60aae6b6cdfe4: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260807 08:50:07.833345 27070 raft_consensus.cc:348] T b8b2b38f85eb452eb7b064fb2875339b P e65182db1a254905939ae7ae864acfd7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dccc17259c6e4d35bcf60aae6b6cdfe4" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 34071 } } peers { permanent_uuid: "e65182db1a254905939ae7ae864acfd7" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 34385 } } peers { permanent_uuid: "9181e852d19c47d2bb93dec238a89ffb" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 46023 } }
I20260807 08:50:07.833513 27070 raft_consensus.cc:374] T b8b2b38f85eb452eb7b064fb2875339b P e65182db1a254905939ae7ae864acfd7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:50:07.833557 27070 raft_consensus.cc:729] T b8b2b38f85eb452eb7b064fb2875339b P e65182db1a254905939ae7ae864acfd7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e65182db1a254905939ae7ae864acfd7, State: Initialized, Role: FOLLOWER
I20260807 08:50:07.833657 27070 consensus_queue.cc:260] T b8b2b38f85eb452eb7b064fb2875339b P e65182db1a254905939ae7ae864acfd7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dccc17259c6e4d35bcf60aae6b6cdfe4" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 34071 } } peers { permanent_uuid: "e65182db1a254905939ae7ae864acfd7" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 34385 } } peers { permanent_uuid: "9181e852d19c47d2bb93dec238a89ffb" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 46023 } }
I20260807 08:50:07.833894 27070 ts_tablet_manager.cc:1428] T b8b2b38f85eb452eb7b064fb2875339b P e65182db1a254905939ae7ae864acfd7: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260807 08:50:07.833945 27056 heartbeater.cc:503] Master 127.4.159.190:34273 was elected leader, sending a full tablet report...
I20260807 08:50:07.894073 27076 raft_consensus.cc:482] T b8b2b38f85eb452eb7b064fb2875339b P e65182db1a254905939ae7ae864acfd7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:50:07.894176 27076 raft_consensus.cc:504] T b8b2b38f85eb452eb7b064fb2875339b P e65182db1a254905939ae7ae864acfd7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dccc17259c6e4d35bcf60aae6b6cdfe4" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 34071 } } peers { permanent_uuid: "e65182db1a254905939ae7ae864acfd7" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 34385 } } peers { permanent_uuid: "9181e852d19c47d2bb93dec238a89ffb" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 46023 } }
I20260807 08:50:07.894533 27076 leader_election.cc:288] T b8b2b38f85eb452eb7b064fb2875339b P e65182db1a254905939ae7ae864acfd7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dccc17259c6e4d35bcf60aae6b6cdfe4 (127.4.159.130:34071), 9181e852d19c47d2bb93dec238a89ffb (127.4.159.129:46023)
I20260807 08:50:07.897326 26665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:07.894721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.131:50949 (local address 127.4.159.129:46023)
0807 08:50:07.894766 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0807 08:50:07.894769 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:50:07.894963 (+ 194us) server_negotiation.cc:407] Connection header received
0807 08:50:07.895022 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:50:07.895023 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:50:07.895033 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:50:07.895081 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:50:07.895233 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:07.895677 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:07.896662 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:07.897028 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:07.897095 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:50:07.897103 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0807 08:50:07.897229 (+ 126us) server_negotiation.cc:299] Negotiation successful
0807 08:50:07.897267 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 08:50:07.897324 27064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:07.894787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.131:43065 (local address 127.4.159.130:34071)
0807 08:50:07.894824 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0807 08:50:07.894826 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:50:07.894834 (+ 8us) server_negotiation.cc:407] Connection header received
0807 08:50:07.894857 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:50:07.894857 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:50:07.894865 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:50:07.894910 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:50:07.895068 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:07.895648 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:07.896599 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:07.896992 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:07.897046 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:50:07.897053 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0807 08:50:07.897210 (+ 157us) server_negotiation.cc:299] Negotiation successful
0807 08:50:07.897249 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 08:50:07.897325 27058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:07.894665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.130:34071 (local address 127.4.159.131:43065)
0807 08:50:07.894719 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:07.894731 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 08:50:07.894796 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:50:07.894937 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:50:07.894938 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:50:07.894939 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:50:07.895008 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:50:07.895009 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:50:07.895719 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:50:07.895720 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:50:07.896515 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:50:07.896518 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:50:07.897062 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:50:07.897063 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:50:07.897139 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:50:07.897142 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 08:50:07.897204 (+ 62us) client_negotiation.cc:240] Negotiation successful
0807 08:50:07.897245 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33,"mutex_wait_us":37}
I20260807 08:50:07.897341 27077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:07.894665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.129:46023 (local address 127.4.159.131:50949)
0807 08:50:07.894905 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:07.894913 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0807 08:50:07.894971 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:50:07.895107 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:50:07.895108 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:50:07.895110 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:50:07.895178 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:50:07.895180 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:50:07.895762 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:50:07.895763 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:50:07.896578 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:50:07.896582 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:50:07.897104 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:50:07.897105 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:50:07.897162 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:50:07.897165 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 08:50:07.897217 (+ 52us) client_negotiation.cc:240] Negotiation successful
0807 08:50:07.897252 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223,"mutex_wait_us":44,"thread_start_us":132,"threads_started":1}
I20260807 08:50:07.897610 26787 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8b2b38f85eb452eb7b064fb2875339b" candidate_uuid: "e65182db1a254905939ae7ae864acfd7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dccc17259c6e4d35bcf60aae6b6cdfe4" is_pre_election: true
I20260807 08:50:07.897737 26787 raft_consensus.cc:2454] T b8b2b38f85eb452eb7b064fb2875339b P dccc17259c6e4d35bcf60aae6b6cdfe4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e65182db1a254905939ae7ae864acfd7 in term 0.
I20260807 08:50:07.897769 26581 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8b2b38f85eb452eb7b064fb2875339b" candidate_uuid: "e65182db1a254905939ae7ae864acfd7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9181e852d19c47d2bb93dec238a89ffb" is_pre_election: true
I20260807 08:50:07.897912 26581 raft_consensus.cc:2454] T b8b2b38f85eb452eb7b064fb2875339b P 9181e852d19c47d2bb93dec238a89ffb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e65182db1a254905939ae7ae864acfd7 in term 0.
I20260807 08:50:07.897961 26894 leader_election.cc:302] T b8b2b38f85eb452eb7b064fb2875339b P e65182db1a254905939ae7ae864acfd7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dccc17259c6e4d35bcf60aae6b6cdfe4, e65182db1a254905939ae7ae864acfd7; no voters:
I20260807 08:50:07.898114 27076 raft_consensus.cc:2781] T b8b2b38f85eb452eb7b064fb2875339b P e65182db1a254905939ae7ae864acfd7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:50:07.898154 27076 raft_consensus.cc:482] T b8b2b38f85eb452eb7b064fb2875339b P e65182db1a254905939ae7ae864acfd7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:50:07.898168 27076 raft_consensus.cc:3037] T b8b2b38f85eb452eb7b064fb2875339b P e65182db1a254905939ae7ae864acfd7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:50:07.898841 27076 raft_consensus.cc:504] T b8b2b38f85eb452eb7b064fb2875339b P e65182db1a254905939ae7ae864acfd7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dccc17259c6e4d35bcf60aae6b6cdfe4" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 34071 } } peers { permanent_uuid: "e65182db1a254905939ae7ae864acfd7" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 34385 } } peers { permanent_uuid: "9181e852d19c47d2bb93dec238a89ffb" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 46023 } }
I20260807 08:50:07.898947 27076 leader_election.cc:288] T b8b2b38f85eb452eb7b064fb2875339b P e65182db1a254905939ae7ae864acfd7 [CANDIDATE]: Term 1 election: Requested vote from peers dccc17259c6e4d35bcf60aae6b6cdfe4 (127.4.159.130:34071), 9181e852d19c47d2bb93dec238a89ffb (127.4.159.129:46023)
I20260807 08:50:07.899088 26581 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8b2b38f85eb452eb7b064fb2875339b" candidate_uuid: "e65182db1a254905939ae7ae864acfd7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9181e852d19c47d2bb93dec238a89ffb"
I20260807 08:50:07.899156 26787 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8b2b38f85eb452eb7b064fb2875339b" candidate_uuid: "e65182db1a254905939ae7ae864acfd7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dccc17259c6e4d35bcf60aae6b6cdfe4"
I20260807 08:50:07.899168 26581 raft_consensus.cc:3037] T b8b2b38f85eb452eb7b064fb2875339b P 9181e852d19c47d2bb93dec238a89ffb [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:50:07.899226 26787 raft_consensus.cc:3037] T b8b2b38f85eb452eb7b064fb2875339b P dccc17259c6e4d35bcf60aae6b6cdfe4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:50:07.900053 26787 raft_consensus.cc:2454] T b8b2b38f85eb452eb7b064fb2875339b P dccc17259c6e4d35bcf60aae6b6cdfe4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e65182db1a254905939ae7ae864acfd7 in term 1.
I20260807 08:50:07.900197 26894 leader_election.cc:302] T b8b2b38f85eb452eb7b064fb2875339b P e65182db1a254905939ae7ae864acfd7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dccc17259c6e4d35bcf60aae6b6cdfe4, e65182db1a254905939ae7ae864acfd7; no voters:
I20260807 08:50:07.900303 27076 raft_consensus.cc:2781] T b8b2b38f85eb452eb7b064fb2875339b P e65182db1a254905939ae7ae864acfd7 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:50:07.900362 26581 raft_consensus.cc:2454] T b8b2b38f85eb452eb7b064fb2875339b P 9181e852d19c47d2bb93dec238a89ffb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e65182db1a254905939ae7ae864acfd7 in term 1.
I20260807 08:50:07.900449 27076 raft_consensus.cc:686] T b8b2b38f85eb452eb7b064fb2875339b P e65182db1a254905939ae7ae864acfd7 [term 1 LEADER]: Becoming Leader. State: Replica: e65182db1a254905939ae7ae864acfd7, State: Running, Role: LEADER
I20260807 08:50:07.900538 27076 consensus_queue.cc:237] T b8b2b38f85eb452eb7b064fb2875339b P e65182db1a254905939ae7ae864acfd7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dccc17259c6e4d35bcf60aae6b6cdfe4" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 34071 } } peers { permanent_uuid: "e65182db1a254905939ae7ae864acfd7" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 34385 } } peers { permanent_uuid: "9181e852d19c47d2bb93dec238a89ffb" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 46023 } }
I20260807 08:50:07.901288 26376 catalog_manager.cc:5495] T b8b2b38f85eb452eb7b064fb2875339b P e65182db1a254905939ae7ae864acfd7 reported cstate change: term changed from 0 to 1, leader changed from <none> to e65182db1a254905939ae7ae864acfd7 (127.4.159.131). New cstate: current_term: 1 leader_uuid: "e65182db1a254905939ae7ae864acfd7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dccc17259c6e4d35bcf60aae6b6cdfe4" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 34071 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e65182db1a254905939ae7ae864acfd7" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 34385 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9181e852d19c47d2bb93dec238a89ffb" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 46023 } health_report { overall_health: UNKNOWN } } }
I20260807 08:50:07.914322 27065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:07.912575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40840 (local address 127.4.159.131:34385)
0807 08:50:07.912610 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0807 08:50:07.912612 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:50:07.912618 (+ 6us) server_negotiation.cc:407] Connection header received
0807 08:50:07.912668 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:50:07.912669 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:50:07.912682 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:50:07.912744 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:50:07.912820 (+ 76us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:07.913217 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:07.913826 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:07.913987 (+ 161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:07.914025 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:50:07.914110 (+ 85us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:50:07.914187 (+ 77us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:50:07.914236 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0807 08:50:07.914249 (+ 13us) server_negotiation.cc:299] Negotiation successful
0807 08:50:07.914262 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260807 08:50:07.915793 26581 raft_consensus.cc:1261] T b8b2b38f85eb452eb7b064fb2875339b P 9181e852d19c47d2bb93dec238a89ffb [term 1 FOLLOWER]: Refusing update from remote peer e65182db1a254905939ae7ae864acfd7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:50:07.915812 26787 raft_consensus.cc:1261] T b8b2b38f85eb452eb7b064fb2875339b P dccc17259c6e4d35bcf60aae6b6cdfe4 [term 1 FOLLOWER]: Refusing update from remote peer e65182db1a254905939ae7ae864acfd7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:50:07.916095 27076 consensus_queue.cc:1035] T b8b2b38f85eb452eb7b064fb2875339b P e65182db1a254905939ae7ae864acfd7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9181e852d19c47d2bb93dec238a89ffb" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 46023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:50:07.916229 27078 consensus_queue.cc:1035] T b8b2b38f85eb452eb7b064fb2875339b P e65182db1a254905939ae7ae864acfd7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dccc17259c6e4d35bcf60aae6b6cdfe4" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 34071 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:50:07.919251 27082 mvcc.cc:205] Tried to move back new op lower bound from 7315835322021076992 to 7315835321961205760. Current Snapshot: MvccSnapshot[applied={T|T < 7315835322021076992 or (T in {7315835322021076992})}]
I20260807 08:50:07.920387 26374 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:59556:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:50:07.920478 26374 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:59556:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:50:07.921114 26374 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7253adae706a42309f5b712541c5b57d: Sending DeleteTablet for 3 replicas of tablet b8b2b38f85eb452eb7b064fb2875339b
I20260807 08:50:07.921434 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 26430
I20260807 08:50:07.921435 26527 tablet_service.cc:1514] Processing DeleteTablet for tablet b8b2b38f85eb452eb7b064fb2875339b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:50:07 UTC) from {username='kudu', principal='kudu/127.4.159.190@KRBTEST.COM'} at 127.0.0.1:60256
I20260807 08:50:07.921434 26752 tablet_service.cc:1514] Processing DeleteTablet for tablet b8b2b38f85eb452eb7b064fb2875339b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:50:07 UTC) from {username='kudu', principal='kudu/127.4.159.190@KRBTEST.COM'} at 127.0.0.1:37414
I20260807 08:50:07.921515 26666 tablet_replica.cc:330] T b8b2b38f85eb452eb7b064fb2875339b P 9181e852d19c47d2bb93dec238a89ffb: stopping tablet replica
I20260807 08:50:07.921555 26666 raft_consensus.cc:2229] T b8b2b38f85eb452eb7b064fb2875339b P 9181e852d19c47d2bb93dec238a89ffb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:50:07.921612 26666 raft_consensus.cc:2258] T b8b2b38f85eb452eb7b064fb2875339b P 9181e852d19c47d2bb93dec238a89ffb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:50:07.932328 26963 tablet_service.cc:1514] Processing DeleteTablet for tablet b8b2b38f85eb452eb7b064fb2875339b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:50:07 UTC) from {username='kudu', principal='kudu/127.4.159.190@KRBTEST.COM'} at 127.0.0.1:40824
I20260807 08:50:07.933028 27093 tablet_replica.cc:330] T b8b2b38f85eb452eb7b064fb2875339b P e65182db1a254905939ae7ae864acfd7: stopping tablet replica
I20260807 08:50:07.933156 27093 raft_consensus.cc:2229] T b8b2b38f85eb452eb7b064fb2875339b P e65182db1a254905939ae7ae864acfd7 [term 1 LEADER]: Raft consensus shutting down.
I20260807 08:50:07.933374 27093 raft_consensus.cc:2258] T b8b2b38f85eb452eb7b064fb2875339b P e65182db1a254905939ae7ae864acfd7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:50:07.934552 27093 ts_tablet_manager.cc:1905] T b8b2b38f85eb452eb7b064fb2875339b P e65182db1a254905939ae7ae864acfd7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:50:07.933058 27092 tablet_replica.cc:330] T b8b2b38f85eb452eb7b064fb2875339b P dccc17259c6e4d35bcf60aae6b6cdfe4: stopping tablet replica
I20260807 08:50:07.934775 27092 raft_consensus.cc:2229] T b8b2b38f85eb452eb7b064fb2875339b P dccc17259c6e4d35bcf60aae6b6cdfe4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:50:07.934923 27092 raft_consensus.cc:2258] T b8b2b38f85eb452eb7b064fb2875339b P dccc17259c6e4d35bcf60aae6b6cdfe4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:50:07.935334 27092 ts_tablet_manager.cc:1905] T b8b2b38f85eb452eb7b064fb2875339b P dccc17259c6e4d35bcf60aae6b6cdfe4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:50:07.936074 27093 ts_tablet_manager.cc:1918] T b8b2b38f85eb452eb7b064fb2875339b P e65182db1a254905939ae7ae864acfd7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:50:07.936146 27093 log.cc:1196] T b8b2b38f85eb452eb7b064fb2875339b P e65182db1a254905939ae7ae864acfd7: Deleting WAL directory at /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-2/wal/wals/b8b2b38f85eb452eb7b064fb2875339b
I20260807 08:50:07.936404 27093 ts_tablet_manager.cc:1939] T b8b2b38f85eb452eb7b064fb2875339b P e65182db1a254905939ae7ae864acfd7: Deleting consensus metadata
I20260807 08:50:07.936767 26352 catalog_manager.cc:4841] TS e65182db1a254905939ae7ae864acfd7 (127.4.159.131:34385): tablet b8b2b38f85eb452eb7b064fb2875339b (table test-table [id=4511ad8077ba4bdeb1be82ce4aabff6b]) successfully deleted
I20260807 08:50:07.937152 27092 ts_tablet_manager.cc:1918] T b8b2b38f85eb452eb7b064fb2875339b P dccc17259c6e4d35bcf60aae6b6cdfe4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:50:07.937225 27092 log.cc:1196] T b8b2b38f85eb452eb7b064fb2875339b P dccc17259c6e4d35bcf60aae6b6cdfe4: Deleting WAL directory at /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-1/wal/wals/b8b2b38f85eb452eb7b064fb2875339b
I20260807 08:50:07.937477 27092 ts_tablet_manager.cc:1939] T b8b2b38f85eb452eb7b064fb2875339b P dccc17259c6e4d35bcf60aae6b6cdfe4: Deleting consensus metadata
I20260807 08:50:07.937855 26351 catalog_manager.cc:4841] TS dccc17259c6e4d35bcf60aae6b6cdfe4 (127.4.159.130:34071): tablet b8b2b38f85eb452eb7b064fb2875339b (table test-table [id=4511ad8077ba4bdeb1be82ce4aabff6b]) successfully deleted
W20260807 08:50:07.939725 26355 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.159.129:46023 (error 108)
I20260807 08:50:07.940114 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 26660
I20260807 08:50:07.940181 27063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:07.940066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.129:46023 (local address 127.0.0.1:60258)
0807 08:50:07.940110 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:07.940151 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.129:46023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260807 08:50:07.945779 26355 catalog_manager.cc:4584] TS 9181e852d19c47d2bb93dec238a89ffb (127.4.159.129:46023): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b8b2b38f85eb452eb7b064fb2875339b: Network error: Client connection negotiation failed: client connection to 127.4.159.129:46023: connect: Connection refused (error 111)
I20260807 08:50:07.958935 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 26864
I20260807 08:50:07.968472 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 26321
I20260807 08:50:07.976478 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.159.190:34273
--webserver_interface=127.4.159.190
--webserver_port=46171
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab
--principal=kudu/127.4.159.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:35687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.159.190:34273
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:50:08.055806 27096 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:50:08.055985 27096 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:50:08.056005 27096 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:50:08.057106 27096 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:50:08.057147 27096 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:50:08.057158 27096 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:50:08.057183 27096 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:50:08.057206 27096 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:50:08.058658 27096 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:35687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.159.190:34273
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab
--principal=kudu/127.4.159.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.159.190:34273
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.159.190
--webserver_port=46171
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27096
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:50:08.058873 27096 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:50:08.059075 27096 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:50:08.063457 27103 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:50:08.063508 27108 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:50:08.063516 27096 server_base.cc:1034] running on GCE node
W20260807 08:50:08.063689 27104 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:50:08.063920 27096 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:50:08.064208 27096 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:50:08.065348 27096 hybrid_clock.cc:648] HybridClock initialized: now 1786092608065339 us; error 27 us; skew 500 ppm
Aug 07 08:50:08 dist-test-slave-6zbq krb5kdc[25496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092608, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:50:08.067641 27096 init.cc:377] Logged in from keytab as kudu/127.4.159.190@KRBTEST.COM (short username kudu)
I20260807 08:50:08.068050 27096 webserver.cc:460] Webserver started at http://127.4.159.190:46171/ using document root <none> and password file <none>
I20260807 08:50:08.068264 27096 fs_manager.cc:359] Metadata directory not provided
I20260807 08:50:08.068306 27096 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:50:08.069554 27096 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:50:08.070313 27117 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:50:08.070487 27096 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:50:08.070568 27096 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/master-0/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/master-0/wal
uuid: "7253adae706a42309f5b712541c5b57d"
format_stamp: "Formatted at 2026-08-07 08:50:06 on dist-test-slave-6zbq"
server_key: "a8bb7a8ab5a147d14a7be7e4363c27a0"
server_key_iv: "e8133f3cd3a115d8ba6b2db3505a635a"
server_key_version: "encryptionkey@0"
I20260807 08:50:08.070869 27096 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:50:08.085861 27096 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:50:08.087610 27096 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:50:08.087734 27096 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:50:08.091559 27096 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.190:34273
I20260807 08:50:08.091596 27195 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.190:34273 every 8 connection(s)
I20260807 08:50:08.091912 27096 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
I20260807 08:50:08.093323 27196 sys_catalog.cc:263] Verifying existing consensus state
I20260807 08:50:08.093952 27196 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7253adae706a42309f5b712541c5b57d: Bootstrap starting.
I20260807 08:50:08.101281 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 27096
I20260807 08:50:08.101610 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.159.129:46023
--local_ip_for_outbound_sockets=127.4.159.129
--tserver_master_addrs=127.4.159.190:34273
--webserver_port=41265
--webserver_interface=127.4.159.129
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.129.keytab
--principal=kudu/127.4.159.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:35687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 08:50:08.102792 27196 log.cc:824] T 00000000000000000000000000000000 P 7253adae706a42309f5b712541c5b57d: Log is configured to *not* fsync() on all Append() calls
I20260807 08:50:08.104909 27196 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7253adae706a42309f5b712541c5b57d: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 08:50:08.105149 27196 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7253adae706a42309f5b712541c5b57d: Bootstrap complete.
I20260807 08:50:08.107282 27196 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7253adae706a42309f5b712541c5b57d [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7253adae706a42309f5b712541c5b57d" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 34273 } }
I20260807 08:50:08.107498 27196 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7253adae706a42309f5b712541c5b57d [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7253adae706a42309f5b712541c5b57d, State: Initialized, Role: FOLLOWER
I20260807 08:50:08.107656 27196 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7253adae706a42309f5b712541c5b57d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7253adae706a42309f5b712541c5b57d" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 34273 } }
I20260807 08:50:08.107729 27196 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7253adae706a42309f5b712541c5b57d [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:50:08.107787 27196 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7253adae706a42309f5b712541c5b57d [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:50:08.107834 27196 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7253adae706a42309f5b712541c5b57d [term 2 FOLLOWER]: Advancing to term 3
I20260807 08:50:08.108762 27196 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7253adae706a42309f5b712541c5b57d [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7253adae706a42309f5b712541c5b57d" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 34273 } }
I20260807 08:50:08.108866 27196 leader_election.cc:302] T 00000000000000000000000000000000 P 7253adae706a42309f5b712541c5b57d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7253adae706a42309f5b712541c5b57d; no voters:
I20260807 08:50:08.109014 27196 leader_election.cc:288] T 00000000000000000000000000000000 P 7253adae706a42309f5b712541c5b57d [CANDIDATE]: Term 3 election: Requested vote from peers
I20260807 08:50:08.109059 27201 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7253adae706a42309f5b712541c5b57d [term 3 FOLLOWER]: Leader election won for term 3
I20260807 08:50:08.109169 27201 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7253adae706a42309f5b712541c5b57d [term 3 LEADER]: Becoming Leader. State: Replica: 7253adae706a42309f5b712541c5b57d, State: Running, Role: LEADER
I20260807 08:50:08.109259 27201 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7253adae706a42309f5b712541c5b57d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7253adae706a42309f5b712541c5b57d" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 34273 } }
I20260807 08:50:08.109264 27196 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7253adae706a42309f5b712541c5b57d [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:50:08.109591 27206 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7253adae706a42309f5b712541c5b57d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "7253adae706a42309f5b712541c5b57d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7253adae706a42309f5b712541c5b57d" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 34273 } } }
I20260807 08:50:08.109601 27208 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7253adae706a42309f5b712541c5b57d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7253adae706a42309f5b712541c5b57d. Latest consensus state: current_term: 3 leader_uuid: "7253adae706a42309f5b712541c5b57d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7253adae706a42309f5b712541c5b57d" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 34273 } } }
I20260807 08:50:08.109683 27206 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7253adae706a42309f5b712541c5b57d [sys.catalog]: This master's current role is: LEADER
I20260807 08:50:08.109690 27208 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7253adae706a42309f5b712541c5b57d [sys.catalog]: This master's current role is: LEADER
I20260807 08:50:08.109907 27213 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:50:08.110733 27213 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:50:08.110852 27213 catalog_manager.cc:1259] Loaded cluster ID: e34fe26867a9489194587342451da3f2
I20260807 08:50:08.111056 27213 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260807 08:50:08.111275 27213 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fdee86159ad google::LogMessage::Fail() at ??:0
@ 0x7fdee861888a google::LogMessage::SendToLog() at ??:0
@ 0x7fdee86154de google::LogMessage::Flush() at ??:0
@ 0x7fdee8616a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fdef05360f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7fdef053f8b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fdeea4e30d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fdeea4d8954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fdeeb33c6db start_thread at ??:0
@ 0x7fdee711361f clone at ??:0
W20260807 08:50:08.194222 27200 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:50:08.194416 27200 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:50:08.194440 27200 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:50:08.195518 27200 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:50:08.195556 27200 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:50:08.195580 27200 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.159.129
I20260807 08:50:08.197022 27200 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:35687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.129.keytab
--principal=kudu/127.4.159.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.159.129:46023
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.159.129
--webserver_port=41265
--tserver_master_addrs=127.4.159.190:34273
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27200
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.159.129
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:50:08.197263 27200 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:50:08.197490 27200 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:50:08.202018 27259 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:50:08.202028 27255 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:50:08.202248 27200 server_base.cc:1034] running on GCE node
W20260807 08:50:08.202026 27256 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:50:08.202466 27200 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:50:08.202687 27200 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:50:08.203846 27200 hybrid_clock.cc:648] HybridClock initialized: now 1786092608203837 us; error 26 us; skew 500 ppm
Aug 07 08:50:08 dist-test-slave-6zbq krb5kdc[25496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092608, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:50:08.206193 27200 init.cc:377] Logged in from keytab as kudu/127.4.159.129@KRBTEST.COM (short username kudu)
I20260807 08:50:08.206616 27200 webserver.cc:460] Webserver started at http://127.4.159.129:41265/ using document root <none> and password file <none>
I20260807 08:50:08.206843 27200 fs_manager.cc:359] Metadata directory not provided
I20260807 08:50:08.206890 27200 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:50:08.208086 27200 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 08:50:08.208673 27268 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:50:08.208853 27200 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 08:50:08.208952 27200 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-0/wal
uuid: "9181e852d19c47d2bb93dec238a89ffb"
format_stamp: "Formatted at 2026-08-07 08:50:06 on dist-test-slave-6zbq"
server_key: "f0e514983f7d1a8dc026d8457e216f4a"
server_key_iv: "7858241c60bdeb4e4b048a12f6a3c95b"
server_key_version: "encryptionkey@0"
I20260807 08:50:08.209357 27200 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:50:08.247957 27200 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:50:08.250128 27200 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:50:08.250284 27200 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:50:08.250542 27200 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:50:08.251017 27280 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260807 08:50:08.252388 27200 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 08:50:08.252446 27200 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260807 08:50:08.252478 27200 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 08:50:08.253334 27200 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 08:50:08.253381 27200 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260807 08:50:08.253432 27280 tablet_bootstrap.cc:492] T b8b2b38f85eb452eb7b064fb2875339b P 9181e852d19c47d2bb93dec238a89ffb: Bootstrap starting.
I20260807 08:50:08.260190 27200 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.129:46023
I20260807 08:50:08.260378 27448 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.129:46023 every 8 connection(s)
I20260807 08:50:08.260609 27200 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-0/data/info.pb
I20260807 08:50:08.263149 27280 log.cc:824] T b8b2b38f85eb452eb7b064fb2875339b P 9181e852d19c47d2bb93dec238a89ffb: Log is configured to *not* fsync() on all Append() calls
I20260807 08:50:08.265127 27280 tablet_bootstrap.cc:492] T b8b2b38f85eb452eb7b064fb2875339b P 9181e852d19c47d2bb93dec238a89ffb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 08:50:08.265374 27280 tablet_bootstrap.cc:492] T b8b2b38f85eb452eb7b064fb2875339b P 9181e852d19c47d2bb93dec238a89ffb: Bootstrap complete.
I20260807 08:50:08.266126 27280 ts_tablet_manager.cc:1397] T b8b2b38f85eb452eb7b064fb2875339b P 9181e852d19c47d2bb93dec238a89ffb: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.008s
I20260807 08:50:08.267091 27280 raft_consensus.cc:348] T b8b2b38f85eb452eb7b064fb2875339b P 9181e852d19c47d2bb93dec238a89ffb [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dccc17259c6e4d35bcf60aae6b6cdfe4" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 34071 } } peers { permanent_uuid: "e65182db1a254905939ae7ae864acfd7" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 34385 } } peers { permanent_uuid: "9181e852d19c47d2bb93dec238a89ffb" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 46023 } }
I20260807 08:50:08.267334 27280 raft_consensus.cc:729] T b8b2b38f85eb452eb7b064fb2875339b P 9181e852d19c47d2bb93dec238a89ffb [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9181e852d19c47d2bb93dec238a89ffb, State: Initialized, Role: FOLLOWER
I20260807 08:50:08.267445 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 27200
I20260807 08:50:08.267457 27280 consensus_queue.cc:260] T b8b2b38f85eb452eb7b064fb2875339b P 9181e852d19c47d2bb93dec238a89ffb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dccc17259c6e4d35bcf60aae6b6cdfe4" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 34071 } } peers { permanent_uuid: "e65182db1a254905939ae7ae864acfd7" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 34385 } } peers { permanent_uuid: "9181e852d19c47d2bb93dec238a89ffb" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 46023 } }
I20260807 08:50:08.267709 27280 ts_tablet_manager.cc:1428] T b8b2b38f85eb452eb7b064fb2875339b P 9181e852d19c47d2bb93dec238a89ffb: Time spent starting tablet: real 0.002s user 0.004s sys 0.000s
I20260807 08:50:08.267727 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.159.130:34071
--local_ip_for_outbound_sockets=127.4.159.130
--tserver_master_addrs=127.4.159.190:34273
--webserver_port=38387
--webserver_interface=127.4.159.130
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.130.keytab
--principal=kudu/127.4.159.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:35687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:50:08.287698 27451 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:50:08.261285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:34273 (local address 127.4.159.129:39489)
0807 08:50:08.261480 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:08.261525 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 08:50:08.262067 (+ 542us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:50:08.287484 (+ 25417us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:34273: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":104,"threads_started":1}
W20260807 08:50:08.287921 27449 heartbeater.cc:644] Failed to heartbeat to 127.4.159.190:34273 (0 consecutive failures): Network error: Failed to ping master at 127.4.159.190:34273: Client connection negotiation failed: client connection to 127.4.159.190:34273: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 08:50:08.288237 27451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:08.288136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:34273 (local address 127.4.159.129:52327)
0807 08:50:08.288181 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:08.288216 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:34273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260807 08:50:08.288548 27451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:08.288486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:34273 (local address 127.4.159.129:53027)
0807 08:50:08.288514 (+ 28us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:08.288534 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:34273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
W20260807 08:50:08.288643 27449 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 08:50:08.350056 27454 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:50:08.350243 27454 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:50:08.350265 27454 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:50:08.351382 27454 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:50:08.351419 27454 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:50:08.351442 27454 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.159.130
I20260807 08:50:08.352864 27454 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:35687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.130.keytab
--principal=kudu/127.4.159.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.159.130:34071
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.159.130
--webserver_port=38387
--tserver_master_addrs=127.4.159.190:34273
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27454
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.159.130
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:50:08.353048 27454 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:50:08.353286 27454 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:50:08.357759 27467 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:50:08.357786 27468 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:50:08.357784 27470 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:50:08.357946 27454 server_base.cc:1034] running on GCE node
I20260807 08:50:08.358157 27454 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:50:08.358413 27454 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:50:08.359553 27454 hybrid_clock.cc:648] HybridClock initialized: now 1786092608359539 us; error 27 us; skew 500 ppm
Aug 07 08:50:08 dist-test-slave-6zbq krb5kdc[25496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092608, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:50:08.361794 27454 init.cc:377] Logged in from keytab as kudu/127.4.159.130@KRBTEST.COM (short username kudu)
I20260807 08:50:08.362145 27454 webserver.cc:460] Webserver started at http://127.4.159.130:38387/ using document root <none> and password file <none>
I20260807 08:50:08.362315 27454 fs_manager.cc:359] Metadata directory not provided
I20260807 08:50:08.362393 27454 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:50:08.363591 27454 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:50:08.364109 27483 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:50:08.364300 27454 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 08:50:08.364399 27454 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-1/wal
uuid: "dccc17259c6e4d35bcf60aae6b6cdfe4"
format_stamp: "Formatted at 2026-08-07 08:50:06 on dist-test-slave-6zbq"
server_key: "f0d427e8337b42861d2f308790acd1f1"
server_key_iv: "d4f7e16f06cac3bc5945628c3ad319eb"
server_key_version: "encryptionkey@0"
I20260807 08:50:08.364706 27454 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:50:08.383625 27454 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:50:08.385185 27454 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:50:08.385298 27454 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:50:08.385507 27454 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:50:08.385783 27454 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:50:08.385821 27454 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:50:08.385841 27454 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:50:08.385864 27454 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:50:08.391258 27454 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.130:34071
I20260807 08:50:08.391319 27650 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.130:34071 every 8 connection(s)
I20260807 08:50:08.391599 27454 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-1/data/info.pb
I20260807 08:50:08.392477 27653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:08.392132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:34273 (local address 127.4.159.130:46939)
0807 08:50:08.392270 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:08.392334 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:34273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":72,"threads_started":1}
W20260807 08:50:08.392832 27651 heartbeater.cc:644] Failed to heartbeat to 127.4.159.190:34273 (0 consecutive failures): Network error: Failed to ping master at 127.4.159.190:34273: Client connection negotiation failed: client connection to 127.4.159.190:34273: connect: Connection refused (error 111)
I20260807 08:50:08.393082 27653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:08.392999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:34273 (local address 127.4.159.130:54479)
0807 08:50:08.393040 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:08.393065 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:34273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260807 08:50:08.393374 27653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:08.393301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:34273 (local address 127.4.159.130:52207)
0807 08:50:08.393336 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:08.393358 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:34273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260807 08:50:08.393460 27651 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 08:50:08.393513 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 27454
I20260807 08:50:08.393730 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.159.131:34385
--local_ip_for_outbound_sockets=127.4.159.131
--tserver_master_addrs=127.4.159.190:34273
--webserver_port=45051
--webserver_interface=127.4.159.131
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.131.keytab
--principal=kudu/127.4.159.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:35687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:50:08.488154 27655 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:50:08.488335 27655 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:50:08.488364 27655 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:50:08.489444 27655 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:50:08.489481 27655 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:50:08.489514 27655 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.159.131
I20260807 08:50:08.490958 27655 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:35687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.131.keytab
--principal=kudu/127.4.159.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.159.131:34385
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.159.131
--webserver_port=45051
--tserver_master_addrs=127.4.159.190:34273
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27655
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.159.131
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:50:08.491173 27655 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:50:08.491369 27655 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:50:08.495821 27665 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:50:08.495875 27668 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:50:08.495829 27663 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:50:08.495877 27655 server_base.cc:1034] running on GCE node
I20260807 08:50:08.496215 27655 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:50:08.496420 27655 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:50:08.497545 27655 hybrid_clock.cc:648] HybridClock initialized: now 1786092608497537 us; error 26 us; skew 500 ppm
Aug 07 08:50:08 dist-test-slave-6zbq krb5kdc[25496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092608, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:50:08.499819 27655 init.cc:377] Logged in from keytab as kudu/127.4.159.131@KRBTEST.COM (short username kudu)
I20260807 08:50:08.500209 27655 webserver.cc:460] Webserver started at http://127.4.159.131:45051/ using document root <none> and password file <none>
I20260807 08:50:08.500392 27655 fs_manager.cc:359] Metadata directory not provided
I20260807 08:50:08.500433 27655 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:50:08.501607 27655 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:50:08.502116 27677 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:50:08.502255 27655 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 08:50:08.502329 27655 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-2/wal
uuid: "e65182db1a254905939ae7ae864acfd7"
format_stamp: "Formatted at 2026-08-07 08:50:07 on dist-test-slave-6zbq"
server_key: "9ef9a62a6ed0d08b081ae4530045c96c"
server_key_iv: "db5536153bf901ec0f6dc22acf5ad28e"
server_key_version: "encryptionkey@0"
I20260807 08:50:08.502628 27655 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:50:08.517627 27655 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:50:08.519650 27655 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:50:08.519771 27655 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:50:08.519960 27655 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:50:08.520236 27655 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:50:08.520267 27655 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:50:08.520287 27655 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:50:08.520309 27655 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:50:08.525516 27655 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.131:34385
I20260807 08:50:08.525555 27858 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.131:34385 every 8 connection(s)
I20260807 08:50:08.525835 27655 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786092453486433-4734-0/minicluster-data/ts-2/data/info.pb
I20260807 08:50:08.526703 27863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:08.526390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:34273 (local address 127.4.159.131:41975)
0807 08:50:08.526522 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:08.526581 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:34273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":69,"threads_started":1}
W20260807 08:50:08.527031 27861 heartbeater.cc:644] Failed to heartbeat to 127.4.159.190:34273 (0 consecutive failures): Network error: Failed to ping master at 127.4.159.190:34273: Client connection negotiation failed: client connection to 127.4.159.190:34273: connect: Connection refused (error 111)
I20260807 08:50:08.527261 27863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:08.527189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:34273 (local address 127.4.159.131:34663)
0807 08:50:08.527214 (+ 25us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:08.527245 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:34273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
I20260807 08:50:08.527447 27863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:08.527401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:34273 (local address 127.4.159.131:42033)
0807 08:50:08.527421 (+ 20us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:08.527436 (+ 15us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:34273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":11}
W20260807 08:50:08.527527 27861 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 08:50:08.528818 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 27655
I20260807 08:50:09.289884 27867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:09.289319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:34273 (local address 127.4.159.129:36087)
0807 08:50:09.289650 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:09.289762 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:34273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":142,"threads_started":1}
I20260807 08:50:09.393989 27868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:09.393775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:34273 (local address 127.4.159.130:40005)
0807 08:50:09.393909 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:09.393943 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:34273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":63,"threads_started":1}
I20260807 08:50:09.482571 27869 raft_consensus.cc:482] T b8b2b38f85eb452eb7b064fb2875339b P 9181e852d19c47d2bb93dec238a89ffb [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:50:09.482713 27869 raft_consensus.cc:504] T b8b2b38f85eb452eb7b064fb2875339b P 9181e852d19c47d2bb93dec238a89ffb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dccc17259c6e4d35bcf60aae6b6cdfe4" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 34071 } } peers { permanent_uuid: "e65182db1a254905939ae7ae864acfd7" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 34385 } } peers { permanent_uuid: "9181e852d19c47d2bb93dec238a89ffb" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 46023 } }
I20260807 08:50:09.483068 27869 leader_election.cc:288] T b8b2b38f85eb452eb7b064fb2875339b P 9181e852d19c47d2bb93dec238a89ffb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers dccc17259c6e4d35bcf60aae6b6cdfe4 (127.4.159.130:34071), e65182db1a254905939ae7ae864acfd7 (127.4.159.131:34385)
Aug 07 08:50:09 dist-test-slave-6zbq krb5kdc[25496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092608, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.129@KRBTEST.COM for kudu/127.4.159.131@KRBTEST.COM
Aug 07 08:50:09 dist-test-slave-6zbq krb5kdc[25496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092608, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.129@KRBTEST.COM for kudu/127.4.159.130@KRBTEST.COM
I20260807 08:50:09.492502 27871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:09.483343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.129:50265 (local address 127.4.159.131:34385)
0807 08:50:09.483510 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0807 08:50:09.483520 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 08:50:09.483540 (+ 20us) server_negotiation.cc:407] Connection header received
0807 08:50:09.483956 (+ 416us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:50:09.483958 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:50:09.484009 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:50:09.484125 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:50:09.484858 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:09.485699 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:09.486710 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:09.487092 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:09.487187 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:50:09.489280 (+ 2093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:50:09.489284 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:50:09.489285 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:50:09.489296 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:50:09.490891 (+ 1595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:50:09.491310 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:50:09.491313 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:50:09.491314 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:50:09.491418 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:50:09.491664 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:50:09.491665 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:50:09.491665 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:50:09.491922 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:50:09.491980 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 08:50:09.492296 (+ 316us) server_negotiation.cc:299] Negotiation successful
0807 08:50:09.492412 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":65,"threads_started":1}
I20260807 08:50:09.492502 27867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:09.483201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.130:34071 (local address 127.4.159.129:41313)
0807 08:50:09.483260 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:09.483272 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 08:50:09.483327 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:50:09.484184 (+ 857us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:50:09.484187 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:50:09.484558 (+ 371us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:50:09.484745 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:50:09.484750 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:50:09.485792 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:50:09.485793 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:50:09.486627 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:50:09.486630 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:50:09.487171 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:50:09.487172 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:50:09.487277 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:50:09.487790 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:50:09.487798 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:50:09.489488 (+ 1690us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:50:09.491222 (+ 1734us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:50:09.491223 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:50:09.491223 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:50:09.491420 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:50:09.491664 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:50:09.491665 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:50:09.491665 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:50:09.491729 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:50:09.492085 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:50:09.492086 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:50:09.492124 (+ 38us) client_negotiation.cc:769] Sending connection context
0807 08:50:09.492211 (+ 87us) client_negotiation.cc:240] Negotiation successful
0807 08:50:09.492389 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32,"mutex_wait_us":45}
I20260807 08:50:09.492569 27870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:09.483234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.131:34385 (local address 127.4.159.129:50265)
0807 08:50:09.483364 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:09.483372 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0807 08:50:09.483407 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:50:09.484217 (+ 810us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:50:09.484218 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:50:09.484560 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:50:09.484744 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:50:09.484750 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:50:09.485808 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:50:09.485808 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:50:09.486626 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:50:09.486628 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:50:09.487170 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:50:09.487171 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:50:09.487275 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:50:09.487839 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:50:09.487843 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:50:09.489162 (+ 1319us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:50:09.490967 (+ 1805us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:50:09.490975 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:50:09.490977 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:50:09.491241 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:50:09.491483 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:50:09.491484 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:50:09.491484 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:50:09.491606 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:50:09.491984 (+ 378us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:50:09.491990 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:50:09.492120 (+ 130us) client_negotiation.cc:769] Sending connection context
0807 08:50:09.492213 (+ 93us) client_negotiation.cc:240] Negotiation successful
0807 08:50:09.492459 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"mutex_wait_us":189,"thread_start_us":71,"threads_started":1}
I20260807 08:50:09.492512 27872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:09.483350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.129:41313 (local address 127.4.159.130:34071)
0807 08:50:09.483511 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0807 08:50:09.483520 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 08:50:09.483549 (+ 29us) server_negotiation.cc:407] Connection header received
0807 08:50:09.483975 (+ 426us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:50:09.483977 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:50:09.484012 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:50:09.484121 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:50:09.484862 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:09.485698 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:09.486704 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:09.487092 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:09.487187 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:50:09.489573 (+ 2386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:50:09.489576 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:50:09.489576 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:50:09.489585 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:50:09.491130 (+ 1545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:50:09.491501 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:50:09.491504 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:50:09.491505 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:50:09.491597 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:50:09.491797 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:50:09.491798 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:50:09.491798 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:50:09.492018 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:50:09.492095 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 08:50:09.492269 (+ 174us) server_negotiation.cc:299] Negotiation successful
0807 08:50:09.492400 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":57,"threads_started":1}
W20260807 08:50:09.493328 27270 leader_election.cc:341] T b8b2b38f85eb452eb7b064fb2875339b P 9181e852d19c47d2bb93dec238a89ffb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e65182db1a254905939ae7ae864acfd7 (127.4.159.131:34385): Not found: Tablet not found: b8b2b38f85eb452eb7b064fb2875339b
W20260807 08:50:09.493523 27269 leader_election.cc:341] T b8b2b38f85eb452eb7b064fb2875339b P 9181e852d19c47d2bb93dec238a89ffb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer dccc17259c6e4d35bcf60aae6b6cdfe4 (127.4.159.130:34071): Not found: Tablet not found: b8b2b38f85eb452eb7b064fb2875339b
I20260807 08:50:09.493594 27269 leader_election.cc:302] T b8b2b38f85eb452eb7b064fb2875339b P 9181e852d19c47d2bb93dec238a89ffb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9181e852d19c47d2bb93dec238a89ffb; no voters: dccc17259c6e4d35bcf60aae6b6cdfe4, e65182db1a254905939ae7ae864acfd7
I20260807 08:50:09.493687 27869 raft_consensus.cc:2726] T b8b2b38f85eb452eb7b064fb2875339b P 9181e852d19c47d2bb93dec238a89ffb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 08:50:09.527938 27874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:09.527783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:34273 (local address 127.4.159.131:51033)
0807 08:50:09.527873 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:09.527909 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:34273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78,"thread_start_us":50,"threads_started":1}
I20260807 08:50:10.290690 27875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:10.290509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:34273 (local address 127.4.159.129:48673)
0807 08:50:10.290617 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:10.290649 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:34273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":62,"threads_started":1}
I20260807 08:50:10.394611 27876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:10.394407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:34273 (local address 127.4.159.130:46013)
0807 08:50:10.394542 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:10.394575 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:34273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":70,"threads_started":1}
I20260807 08:50:10.528460 27878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:10.528260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:34273 (local address 127.4.159.131:48853)
0807 08:50:10.528394 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:10.528426 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:34273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":73,"threads_started":1}
I20260807 08:50:11.291249 27880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:11.291047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:34273 (local address 127.4.159.129:33095)
0807 08:50:11.291178 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:11.291207 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:34273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":76,"threads_started":1}
I20260807 08:50:11.395180 27881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:11.394972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:34273 (local address 127.4.159.130:40273)
0807 08:50:11.395105 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:11.395139 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:34273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":78,"threads_started":1}
I20260807 08:50:11.529003 27882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:11.528822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:34273 (local address 127.4.159.131:58079)
0807 08:50:11.528934 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:11.528965 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:34273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":61,"threads_started":1}
I20260807 08:50:11.581314 27883 raft_consensus.cc:482] T b8b2b38f85eb452eb7b064fb2875339b P 9181e852d19c47d2bb93dec238a89ffb [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:50:11.581403 27883 raft_consensus.cc:504] T b8b2b38f85eb452eb7b064fb2875339b P 9181e852d19c47d2bb93dec238a89ffb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dccc17259c6e4d35bcf60aae6b6cdfe4" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 34071 } } peers { permanent_uuid: "e65182db1a254905939ae7ae864acfd7" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 34385 } } peers { permanent_uuid: "9181e852d19c47d2bb93dec238a89ffb" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 46023 } }
I20260807 08:50:11.581537 27883 leader_election.cc:288] T b8b2b38f85eb452eb7b064fb2875339b P 9181e852d19c47d2bb93dec238a89ffb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers dccc17259c6e4d35bcf60aae6b6cdfe4 (127.4.159.130:34071), e65182db1a254905939ae7ae864acfd7 (127.4.159.131:34385)
W20260807 08:50:11.581933 27270 leader_election.cc:341] T b8b2b38f85eb452eb7b064fb2875339b P 9181e852d19c47d2bb93dec238a89ffb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e65182db1a254905939ae7ae864acfd7 (127.4.159.131:34385): Not found: Tablet not found: b8b2b38f85eb452eb7b064fb2875339b
W20260807 08:50:11.582050 27269 leader_election.cc:341] T b8b2b38f85eb452eb7b064fb2875339b P 9181e852d19c47d2bb93dec238a89ffb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer dccc17259c6e4d35bcf60aae6b6cdfe4 (127.4.159.130:34071): Not found: Tablet not found: b8b2b38f85eb452eb7b064fb2875339b
I20260807 08:50:11.582118 27269 leader_election.cc:302] T b8b2b38f85eb452eb7b064fb2875339b P 9181e852d19c47d2bb93dec238a89ffb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9181e852d19c47d2bb93dec238a89ffb; no voters: dccc17259c6e4d35bcf60aae6b6cdfe4, e65182db1a254905939ae7ae864acfd7
I20260807 08:50:11.582197 27883 raft_consensus.cc:2726] T b8b2b38f85eb452eb7b064fb2875339b P 9181e852d19c47d2bb93dec238a89ffb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 08:50:12.291832 27884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:12.291636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:34273 (local address 127.4.159.129:60415)
0807 08:50:12.291752 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:12.291785 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:34273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":60,"threads_started":1}
I20260807 08:50:12.395715 27885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:12.395521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:34273 (local address 127.4.159.130:53091)
0807 08:50:12.395649 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:12.395680 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:34273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":74,"threads_started":1}
I20260807 08:50:12.529544 27886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:12.529354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:34273 (local address 127.4.159.131:53767)
0807 08:50:12.529481 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:12.529511 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:34273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":76,"threads_started":1}
I20260807 08:50:13.292399 27887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:13.292216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:34273 (local address 127.4.159.129:37375)
0807 08:50:13.292325 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:13.292359 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:34273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":55,"threads_started":1}
I20260807 08:50:13.396250 27888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:13.396045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:34273 (local address 127.4.159.130:56081)
0807 08:50:13.396181 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:13.396214 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:34273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":88,"threads_started":1}
I20260807 08:50:13.530100 27889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:13.529911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:34273 (local address 127.4.159.131:34801)
0807 08:50:13.530028 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:13.530062 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:34273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":58,"threads_started":1}
I20260807 08:50:14.292943 27891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:14.292757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:34273 (local address 127.4.159.129:38433)
0807 08:50:14.292870 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:14.292901 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:34273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":59,"threads_started":1}
I20260807 08:50:14.396839 27893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:14.396651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:34273 (local address 127.4.159.130:56573)
0807 08:50:14.396766 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:14.396800 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:34273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":61,"threads_started":1}
I20260807 08:50:14.503334 27894 raft_consensus.cc:482] T b8b2b38f85eb452eb7b064fb2875339b P 9181e852d19c47d2bb93dec238a89ffb [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:50:14.503422 27894 raft_consensus.cc:504] T b8b2b38f85eb452eb7b064fb2875339b P 9181e852d19c47d2bb93dec238a89ffb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dccc17259c6e4d35bcf60aae6b6cdfe4" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 34071 } } peers { permanent_uuid: "e65182db1a254905939ae7ae864acfd7" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 34385 } } peers { permanent_uuid: "9181e852d19c47d2bb93dec238a89ffb" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 46023 } }
I20260807 08:50:14.503588 27894 leader_election.cc:288] T b8b2b38f85eb452eb7b064fb2875339b P 9181e852d19c47d2bb93dec238a89ffb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers dccc17259c6e4d35bcf60aae6b6cdfe4 (127.4.159.130:34071), e65182db1a254905939ae7ae864acfd7 (127.4.159.131:34385)
W20260807 08:50:14.503908 27269 leader_election.cc:341] T b8b2b38f85eb452eb7b064fb2875339b P 9181e852d19c47d2bb93dec238a89ffb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer dccc17259c6e4d35bcf60aae6b6cdfe4 (127.4.159.130:34071): Not found: Tablet not found: b8b2b38f85eb452eb7b064fb2875339b
W20260807 08:50:14.503986 27270 leader_election.cc:341] T b8b2b38f85eb452eb7b064fb2875339b P 9181e852d19c47d2bb93dec238a89ffb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e65182db1a254905939ae7ae864acfd7 (127.4.159.131:34385): Not found: Tablet not found: b8b2b38f85eb452eb7b064fb2875339b
I20260807 08:50:14.504055 27270 leader_election.cc:302] T b8b2b38f85eb452eb7b064fb2875339b P 9181e852d19c47d2bb93dec238a89ffb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9181e852d19c47d2bb93dec238a89ffb; no voters: dccc17259c6e4d35bcf60aae6b6cdfe4, e65182db1a254905939ae7ae864acfd7
I20260807 08:50:14.504137 27894 raft_consensus.cc:2726] T b8b2b38f85eb452eb7b064fb2875339b P 9181e852d19c47d2bb93dec238a89ffb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 08:50:14.530696 27897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:14.530499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:34273 (local address 127.4.159.131:37329)
0807 08:50:14.530615 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:14.530653 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:34273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":58,"threads_started":1}
I20260807 08:50:15.293646 27898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:15.293414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:34273 (local address 127.4.159.129:49309)
0807 08:50:15.293560 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:15.293597 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:34273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":66,"threads_started":1}
I20260807 08:50:15.397451 27899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:15.397263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:34273 (local address 127.4.159.130:58349)
0807 08:50:15.397380 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:15.397414 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:34273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":62,"threads_started":1}
I20260807 08:50:15.531464 27900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:15.531206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:34273 (local address 127.4.159.131:35419)
0807 08:50:15.531356 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:15.531405 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:34273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":85,"threads_started":1}
I20260807 08:50:16.294324 27902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:16.294075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:34273 (local address 127.4.159.129:42851)
0807 08:50:16.294241 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:16.294277 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:34273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":66,"threads_started":1}
I20260807 08:50:16.398036 27904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:16.397833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:34273 (local address 127.4.159.130:42649)
0807 08:50:16.397964 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:16.397997 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:34273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":71,"threads_started":1}
I20260807 08:50:16.532299 27905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:16.532019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:34273 (local address 127.4.159.131:55713)
0807 08:50:16.532174 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:16.532244 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:34273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":69,"threads_started":1}
I20260807 08:50:17.295105 27906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:17.294874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:34273 (local address 127.4.159.129:56767)
0807 08:50:17.295002 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:17.295050 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:34273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":69,"threads_started":1}
I20260807 08:50:17.398837 27907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:17.398562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:34273 (local address 127.4.159.130:55583)
0807 08:50:17.398740 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:17.398792 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:34273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":67,"threads_started":1}
I20260807 08:50:17.532985 27910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:17.532777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:34273 (local address 127.4.159.131:37997)
0807 08:50:17.532906 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:17.532947 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:34273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":61,"threads_started":1}
I20260807 08:50:17.595934 27911 raft_consensus.cc:482] T b8b2b38f85eb452eb7b064fb2875339b P 9181e852d19c47d2bb93dec238a89ffb [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:50:17.596033 27911 raft_consensus.cc:504] T b8b2b38f85eb452eb7b064fb2875339b P 9181e852d19c47d2bb93dec238a89ffb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dccc17259c6e4d35bcf60aae6b6cdfe4" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 34071 } } peers { permanent_uuid: "e65182db1a254905939ae7ae864acfd7" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 34385 } } peers { permanent_uuid: "9181e852d19c47d2bb93dec238a89ffb" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 46023 } }
I20260807 08:50:17.596208 27911 leader_election.cc:288] T b8b2b38f85eb452eb7b064fb2875339b P 9181e852d19c47d2bb93dec238a89ffb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers dccc17259c6e4d35bcf60aae6b6cdfe4 (127.4.159.130:34071), e65182db1a254905939ae7ae864acfd7 (127.4.159.131:34385)
W20260807 08:50:17.596589 27269 leader_election.cc:341] T b8b2b38f85eb452eb7b064fb2875339b P 9181e852d19c47d2bb93dec238a89ffb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer dccc17259c6e4d35bcf60aae6b6cdfe4 (127.4.159.130:34071): Not found: Tablet not found: b8b2b38f85eb452eb7b064fb2875339b
W20260807 08:50:17.596683 27270 leader_election.cc:341] T b8b2b38f85eb452eb7b064fb2875339b P 9181e852d19c47d2bb93dec238a89ffb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e65182db1a254905939ae7ae864acfd7 (127.4.159.131:34385): Not found: Tablet not found: b8b2b38f85eb452eb7b064fb2875339b
I20260807 08:50:17.596725 27270 leader_election.cc:302] T b8b2b38f85eb452eb7b064fb2875339b P 9181e852d19c47d2bb93dec238a89ffb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9181e852d19c47d2bb93dec238a89ffb; no voters: dccc17259c6e4d35bcf60aae6b6cdfe4, e65182db1a254905939ae7ae864acfd7
I20260807 08:50:17.596800 27911 raft_consensus.cc:2726] T b8b2b38f85eb452eb7b064fb2875339b P 9181e852d19c47d2bb93dec238a89ffb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 08:50:18.295748 27912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:18.295546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:34273 (local address 127.4.159.129:57671)
0807 08:50:18.295662 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:18.295699 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:34273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":70,"threads_started":1}
I20260807 08:50:18.399518 27913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:18.399308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:34273 (local address 127.4.159.130:36105)
0807 08:50:18.399449 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:18.399482 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:34273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":78,"threads_started":1}
I20260807 08:50:18.533524 27914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:18.533340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:34273 (local address 127.4.159.131:34381)
0807 08:50:18.533459 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:18.533487 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:34273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":68,"threads_started":1}
I20260807 08:50:19.296298 27915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:19.296109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:34273 (local address 127.4.159.129:46739)
0807 08:50:19.296220 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:19.296253 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:34273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":64,"threads_started":1}
I20260807 08:50:19.400075 27916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:19.399892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:34273 (local address 127.4.159.130:32915)
0807 08:50:19.400005 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:19.400038 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:34273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":56,"threads_started":1}
I20260807 08:50:19.534109 27917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:19.533904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:34273 (local address 127.4.159.131:53015)
0807 08:50:19.534025 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:19.534061 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:34273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":55,"threads_started":1}
I20260807 08:50:20.295692 27918 raft_consensus.cc:482] T b8b2b38f85eb452eb7b064fb2875339b P 9181e852d19c47d2bb93dec238a89ffb [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:50:20.295781 27918 raft_consensus.cc:504] T b8b2b38f85eb452eb7b064fb2875339b P 9181e852d19c47d2bb93dec238a89ffb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dccc17259c6e4d35bcf60aae6b6cdfe4" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 34071 } } peers { permanent_uuid: "e65182db1a254905939ae7ae864acfd7" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 34385 } } peers { permanent_uuid: "9181e852d19c47d2bb93dec238a89ffb" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 46023 } }
I20260807 08:50:20.295919 27918 leader_election.cc:288] T b8b2b38f85eb452eb7b064fb2875339b P 9181e852d19c47d2bb93dec238a89ffb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers dccc17259c6e4d35bcf60aae6b6cdfe4 (127.4.159.130:34071), e65182db1a254905939ae7ae864acfd7 (127.4.159.131:34385)
W20260807 08:50:20.296299 27269 leader_election.cc:341] T b8b2b38f85eb452eb7b064fb2875339b P 9181e852d19c47d2bb93dec238a89ffb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer dccc17259c6e4d35bcf60aae6b6cdfe4 (127.4.159.130:34071): Not found: Tablet not found: b8b2b38f85eb452eb7b064fb2875339b
W20260807 08:50:20.296391 27270 leader_election.cc:341] T b8b2b38f85eb452eb7b064fb2875339b P 9181e852d19c47d2bb93dec238a89ffb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e65182db1a254905939ae7ae864acfd7 (127.4.159.131:34385): Not found: Tablet not found: b8b2b38f85eb452eb7b064fb2875339b
I20260807 08:50:20.296459 27270 leader_election.cc:302] T b8b2b38f85eb452eb7b064fb2875339b P 9181e852d19c47d2bb93dec238a89ffb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9181e852d19c47d2bb93dec238a89ffb; no voters: dccc17259c6e4d35bcf60aae6b6cdfe4, e65182db1a254905939ae7ae864acfd7
I20260807 08:50:20.296558 27918 raft_consensus.cc:2726] T b8b2b38f85eb452eb7b064fb2875339b P 9181e852d19c47d2bb93dec238a89ffb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 08:50:20.296834 27920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:20.296656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:34273 (local address 127.4.159.129:46661)
0807 08:50:20.296771 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:20.296801 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:34273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":69,"threads_started":1}
I20260807 08:50:20.400738 27921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:20.400482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:34273 (local address 127.4.159.130:48483)
0807 08:50:20.400652 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:20.400695 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:34273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":84,"threads_started":1}
I20260807 08:50:20.534737 27922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:20.534544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:34273 (local address 127.4.159.131:53933)
0807 08:50:20.534654 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:20.534688 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:34273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":59,"threads_started":1}
I20260807 08:50:21.297367 27923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:21.297181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:34273 (local address 127.4.159.129:47727)
0807 08:50:21.297292 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:21.297323 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:34273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":60,"threads_started":1}
I20260807 08:50:21.401397 27928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:21.401207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:34273 (local address 127.4.159.130:45833)
0807 08:50:21.401329 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:21.401358 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:34273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":60,"threads_started":1}
I20260807 08:50:21.535331 27929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:21.535149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:34273 (local address 127.4.159.131:45053)
0807 08:50:21.535257 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:21.535291 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:34273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":57,"threads_started":1}
I20260807 08:50:22.297909 27930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:22.297720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:34273 (local address 127.4.159.129:54445)
0807 08:50:22.297829 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:22.297862 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:34273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":59,"threads_started":1}
I20260807 08:50:22.401907 27932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:22.401741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:34273 (local address 127.4.159.130:47977)
0807 08:50:22.401847 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:22.401871 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:34273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":55,"threads_started":1}
I20260807 08:50:22.535892 27933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:22.535715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:34273 (local address 127.4.159.131:44067)
0807 08:50:22.535824 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:22.535856 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:34273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":61,"threads_started":1}
I20260807 08:50:23.298560 27934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:23.298375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:34273 (local address 127.4.159.129:56781)
0807 08:50:23.298484 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:23.298515 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:34273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":59,"threads_started":1}
I20260807 08:50:23.402603 27936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:23.402407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:34273 (local address 127.4.159.130:46573)
0807 08:50:23.402530 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:23.402563 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:34273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":74,"threads_started":1}
I20260807 08:50:23.536482 27937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:23.536293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:34273 (local address 127.4.159.131:55579)
0807 08:50:23.536413 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:23.536444 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:34273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":59,"threads_started":1}
I20260807 08:50:23.538331 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 27200
I20260807 08:50:23.551944 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 27454
I20260807 08:50:23.560705 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 27655
2026-08-07T08:50:23Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18242 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:50:23 dist-test-slave-6zbq krb5kdc[27941](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:50:23 dist-test-slave-6zbq krb5kdc[27941](info): set up 2 sockets
Aug 07 08:50:23 dist-test-slave-6zbq krb5kdc[27941](info): commencing operation
krb5kdc: starting...
W20260807 08:50:24.905230 4734 mini_kdc.cc:121] Time spent starting KDC: real 1.320s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/krb5kdc/test-user.keytab.
Aug 07 08:50:25 dist-test-slave-6zbq krb5kdc[27941](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092625, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:50:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:50:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.159.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.159.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.159.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab.
Entry for principal kudu/127.4.159.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab.
I20260807 08:50:25.117445 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.159.190:42791
--webserver_interface=127.4.159.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab
--principal=kudu/127.4.159.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:33467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.159.190:42791
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:50:25.334905 27960 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:50:25.335412 27960 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:50:25.335479 27960 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:50:25.338160 27960 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 08:50:25.338244 27960 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 08:50:25.338281 27960 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 08:50:25.338407 27960 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:50:25.338457 27960 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:50:25.338538 27960 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:50:25.338610 27960 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:50:25.338680 27960 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:50:25.342427 27960 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:33467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.4.159.190:42791
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab
--principal=kudu/127.4.159.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.159.190:42791
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.159.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27960
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:50:25.343235 27960 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:50:25.343920 27960 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:50:25.354892 27969 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:50:25.354861 27965 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:50:25.354871 27966 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:50:25.355356 27960 server_base.cc:1034] running on GCE node
I20260807 08:50:25.356279 27960 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:50:25.357048 27960 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:50:25.358314 27960 hybrid_clock.cc:648] HybridClock initialized: now 1786092625358286 us; error 51 us; skew 500 ppm
Aug 07 08:50:25 dist-test-slave-6zbq krb5kdc[27941](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092625, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:50:25.363986 27960 init.cc:377] Logged in from keytab as kudu/127.4.159.190@KRBTEST.COM (short username kudu)
I20260807 08:50:25.364902 27960 webserver.cc:460] Webserver started at http://127.4.159.190:33511/ using document root <none> and password file <none>
I20260807 08:50:25.365427 27960 fs_manager.cc:359] Metadata directory not provided
I20260807 08:50:25.365552 27960 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:50:25.365790 27960 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:50:25.368666 27960 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/master-0/data/instance:
uuid: "d418769ff6aa466ea11935ea8f4077d0"
format_stamp: "Formatted at 2026-08-07 08:50:25 on dist-test-slave-6zbq"
server_key: "6cf8a20136614e6d7285cef1f0474d00"
server_key_iv: "601392ea180b0a4217ec5d7c962a222b"
server_key_version: "encryptionkey@0"
I20260807 08:50:25.369613 27960 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/master-0/wal/instance:
uuid: "d418769ff6aa466ea11935ea8f4077d0"
format_stamp: "Formatted at 2026-08-07 08:50:25 on dist-test-slave-6zbq"
server_key: "6cf8a20136614e6d7285cef1f0474d00"
server_key_iv: "601392ea180b0a4217ec5d7c962a222b"
server_key_version: "encryptionkey@0"
I20260807 08:50:25.374488 27960 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260807 08:50:25.376329 27977 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:50:25.376682 27960 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20260807 08:50:25.376901 27960 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/master-0/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/master-0/wal
uuid: "d418769ff6aa466ea11935ea8f4077d0"
format_stamp: "Formatted at 2026-08-07 08:50:25 on dist-test-slave-6zbq"
server_key: "6cf8a20136614e6d7285cef1f0474d00"
server_key_iv: "601392ea180b0a4217ec5d7c962a222b"
server_key_version: "encryptionkey@0"
I20260807 08:50:25.377048 27960 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:50:25.416538 27960 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:50:25.418403 27960 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:50:25.418537 27960 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:50:25.422431 27960 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.190:42791
I20260807 08:50:25.422492 28050 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.190:42791 every 8 connection(s)
I20260807 08:50:25.422804 27960 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
I20260807 08:50:25.423589 28052 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:50:25.425796 28052 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d418769ff6aa466ea11935ea8f4077d0: Bootstrap starting.
I20260807 08:50:25.426383 28052 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d418769ff6aa466ea11935ea8f4077d0: Neither blocks nor log segments found. Creating new log.
I20260807 08:50:25.426661 28052 log.cc:824] T 00000000000000000000000000000000 P d418769ff6aa466ea11935ea8f4077d0: Log is configured to *not* fsync() on all Append() calls
I20260807 08:50:25.427330 28052 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d418769ff6aa466ea11935ea8f4077d0: No bootstrap required, opened a new log
I20260807 08:50:25.428862 28052 raft_consensus.cc:348] T 00000000000000000000000000000000 P d418769ff6aa466ea11935ea8f4077d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d418769ff6aa466ea11935ea8f4077d0" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 42791 } }
I20260807 08:50:25.428995 28052 raft_consensus.cc:374] T 00000000000000000000000000000000 P d418769ff6aa466ea11935ea8f4077d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:50:25.429016 28052 raft_consensus.cc:729] T 00000000000000000000000000000000 P d418769ff6aa466ea11935ea8f4077d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d418769ff6aa466ea11935ea8f4077d0, State: Initialized, Role: FOLLOWER
I20260807 08:50:25.429137 28052 consensus_queue.cc:260] T 00000000000000000000000000000000 P d418769ff6aa466ea11935ea8f4077d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d418769ff6aa466ea11935ea8f4077d0" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 42791 } }
I20260807 08:50:25.429226 28052 raft_consensus.cc:388] T 00000000000000000000000000000000 P d418769ff6aa466ea11935ea8f4077d0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:50:25.429272 28052 raft_consensus.cc:482] T 00000000000000000000000000000000 P d418769ff6aa466ea11935ea8f4077d0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:50:25.429328 28052 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d418769ff6aa466ea11935ea8f4077d0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:50:25.429342 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 27960
I20260807 08:50:25.429430 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/master-0/wal/instance
I20260807 08:50:25.429633 4734 external_mini_cluster.cc:1428] Setting key 46d2882b1c4b644758afe4dbda6d672a
I20260807 08:50:25.429968 28052 raft_consensus.cc:504] T 00000000000000000000000000000000 P d418769ff6aa466ea11935ea8f4077d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d418769ff6aa466ea11935ea8f4077d0" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 42791 } }
I20260807 08:50:25.430071 28052 leader_election.cc:302] T 00000000000000000000000000000000 P d418769ff6aa466ea11935ea8f4077d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d418769ff6aa466ea11935ea8f4077d0; no voters:
I20260807 08:50:25.430265 28052 leader_election.cc:288] T 00000000000000000000000000000000 P d418769ff6aa466ea11935ea8f4077d0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:50:25.430543 28058 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d418769ff6aa466ea11935ea8f4077d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:50:25.430578 28052 sys_catalog.cc:564] T 00000000000000000000000000000000 P d418769ff6aa466ea11935ea8f4077d0 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:50:25.430668 28058 raft_consensus.cc:686] T 00000000000000000000000000000000 P d418769ff6aa466ea11935ea8f4077d0 [term 1 LEADER]: Becoming Leader. State: Replica: d418769ff6aa466ea11935ea8f4077d0, State: Running, Role: LEADER
I20260807 08:50:25.430830 28058 consensus_queue.cc:237] T 00000000000000000000000000000000 P d418769ff6aa466ea11935ea8f4077d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d418769ff6aa466ea11935ea8f4077d0" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 42791 } }
I20260807 08:50:25.431207 28058 sys_catalog.cc:455] T 00000000000000000000000000000000 P d418769ff6aa466ea11935ea8f4077d0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d418769ff6aa466ea11935ea8f4077d0. Latest consensus state: current_term: 1 leader_uuid: "d418769ff6aa466ea11935ea8f4077d0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d418769ff6aa466ea11935ea8f4077d0" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 42791 } } }
I20260807 08:50:25.431353 28058 sys_catalog.cc:458] T 00000000000000000000000000000000 P d418769ff6aa466ea11935ea8f4077d0 [sys.catalog]: This master's current role is: LEADER
I20260807 08:50:25.431593 28074 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:50:25.431572 28059 sys_catalog.cc:455] T 00000000000000000000000000000000 P d418769ff6aa466ea11935ea8f4077d0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d418769ff6aa466ea11935ea8f4077d0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d418769ff6aa466ea11935ea8f4077d0" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 42791 } } }
I20260807 08:50:25.431789 28059 sys_catalog.cc:458] T 00000000000000000000000000000000 P d418769ff6aa466ea11935ea8f4077d0 [sys.catalog]: This master's current role is: LEADER
I20260807 08:50:25.432150 28074 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:50:25.433668 28074 catalog_manager.cc:1347] Generated new cluster ID: 4ab036fec3ac4624825127d7f6e296ab
I20260807 08:50:25.433724 28074 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 08:50:25 dist-test-slave-6zbq krb5kdc[27941](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092625, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.159.190@KRBTEST.COM
I20260807 08:50:25.440505 28057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:25.430100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43070 (local address 127.4.159.190:42791)
0807 08:50:25.430297 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0807 08:50:25.430305 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 08:50:25.430354 (+ 49us) server_negotiation.cc:407] Connection header received
0807 08:50:25.430495 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:50:25.430502 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:50:25.430557 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:50:25.430733 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:50:25.431390 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:25.432807 (+ 1417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:25.433816 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:25.434432 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:25.434519 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:50:25.437012 (+ 2493us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:50:25.437016 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:50:25.437017 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:50:25.437028 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:50:25.438785 (+ 1757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:50:25.439126 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:50:25.439129 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:50:25.439129 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:50:25.439220 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:50:25.439385 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:50:25.439386 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:50:25.439386 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:50:25.439646 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:50:25.439772 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0807 08:50:25.440157 (+ 385us) server_negotiation.cc:299] Negotiation successful
0807 08:50:25.440284 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":131,"thread_start_us":102,"threads_started":1}
I20260807 08:50:25.453531 28074 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:50:25.453987 28074 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:50:25.462132 28074 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d418769ff6aa466ea11935ea8f4077d0: Generated new TSK 0
I20260807 08:50:25.462283 28074 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.159.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.159.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.159.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.129.keytab.
Entry for principal kudu/127.4.159.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.129.keytab.
I20260807 08:50:25.493659 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.159.129:0
--local_ip_for_outbound_sockets=127.4.159.129
--webserver_interface=127.4.159.129
--webserver_port=0
--tserver_master_addrs=127.4.159.190:42791
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.129.keytab
--principal=kudu/127.4.159.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:33467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:50:25.589507 28092 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:50:25.589699 28092 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:50:25.589721 28092 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:50:25.590803 28092 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:50:25.590840 28092 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:50:25.590865 28092 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.159.129
I20260807 08:50:25.592227 28092 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:33467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.129.keytab
--principal=kudu/127.4.159.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.159.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.159.129
--webserver_port=0
--tserver_master_addrs=127.4.159.190:42791
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28092
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.159.129
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:50:25.592406 28092 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:50:25.592567 28092 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:50:25.597065 28101 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:50:25.597051 28098 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:50:25.597115 28099 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:50:25.597257 28092 server_base.cc:1034] running on GCE node
I20260807 08:50:25.597424 28092 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:50:25.597600 28092 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:50:25.598752 28092 hybrid_clock.cc:648] HybridClock initialized: now 1786092625598738 us; error 26 us; skew 500 ppm
Aug 07 08:50:25 dist-test-slave-6zbq krb5kdc[27941](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092625, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:50:25.601121 28092 init.cc:377] Logged in from keytab as kudu/127.4.159.129@KRBTEST.COM (short username kudu)
I20260807 08:50:25.601496 28092 webserver.cc:460] Webserver started at http://127.4.159.129:35191/ using document root <none> and password file <none>
I20260807 08:50:25.601689 28092 fs_manager.cc:359] Metadata directory not provided
I20260807 08:50:25.601744 28092 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:50:25.601866 28092 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:50:25.602846 28092 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-0/data/instance:
uuid: "0a8f9e196ee740fe9fdf8e9b935607d2"
format_stamp: "Formatted at 2026-08-07 08:50:25 on dist-test-slave-6zbq"
server_key: "682ec57daf6455a5f60b2065bed6d968"
server_key_iv: "4d2c0c2ea6783ca8e60a54dc0d34f9a9"
server_key_version: "encryptionkey@0"
I20260807 08:50:25.603158 28092 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-0/wal/instance:
uuid: "0a8f9e196ee740fe9fdf8e9b935607d2"
format_stamp: "Formatted at 2026-08-07 08:50:25 on dist-test-slave-6zbq"
server_key: "682ec57daf6455a5f60b2065bed6d968"
server_key_iv: "4d2c0c2ea6783ca8e60a54dc0d34f9a9"
server_key_version: "encryptionkey@0"
I20260807 08:50:25.604338 28092 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 08:50:25.604960 28114 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:50:25.605096 28092 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 08:50:25.605176 28092 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-0/wal
uuid: "0a8f9e196ee740fe9fdf8e9b935607d2"
format_stamp: "Formatted at 2026-08-07 08:50:25 on dist-test-slave-6zbq"
server_key: "682ec57daf6455a5f60b2065bed6d968"
server_key_iv: "4d2c0c2ea6783ca8e60a54dc0d34f9a9"
server_key_version: "encryptionkey@0"
I20260807 08:50:25.605234 28092 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:50:25.649262 28092 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:50:25.650971 28092 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:50:25.651100 28092 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:50:25.651301 28092 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:50:25.651574 28092 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:50:25.651607 28092 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:50:25.651635 28092 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:50:25.651654 28092 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:50:25.656894 28092 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.129:42675
I20260807 08:50:25.656955 28281 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.129:42675 every 8 connection(s)
I20260807 08:50:25.657214 28092 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-0/data/info.pb
I20260807 08:50:25.658649 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 28092
I20260807 08:50:25.658756 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-0/wal/instance
I20260807 08:50:25.658957 4734 external_mini_cluster.cc:1428] Setting key 4204ef57854e7f8fdc210a4f94fcf342
Aug 07 08:50:25 dist-test-slave-6zbq krb5kdc[27941](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092625, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.129@KRBTEST.COM for kudu/127.4.159.190@KRBTEST.COM
I20260807 08:50:25.666440 28057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:25.657845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.129:44817 (local address 127.4.159.190:42791)
0807 08:50:25.657889 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0807 08:50:25.657890 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 08:50:25.658063 (+ 173us) server_negotiation.cc:407] Connection header received
0807 08:50:25.658582 (+ 519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:50:25.658583 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:50:25.658595 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:50:25.658662 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:50:25.659781 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:25.660433 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:25.661377 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:25.661674 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:25.661745 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:50:25.663632 (+ 1887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:50:25.663634 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:50:25.663634 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:50:25.663645 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:50:25.665177 (+ 1532us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:50:25.665537 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:50:25.665538 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:50:25.665539 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:50:25.665604 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:50:25.665810 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:50:25.665811 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:50:25.665812 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:50:25.665947 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:50:25.666019 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 08:50:25.666231 (+ 212us) server_negotiation.cc:299] Negotiation successful
0807 08:50:25.666320 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260807 08:50:25.666489 28285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:25.657806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:42791 (local address 127.4.159.129:44817)
0807 08:50:25.657958 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:25.657997 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 08:50:25.658493 (+ 496us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:50:25.658732 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:50:25.658736 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:50:25.659435 (+ 699us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:50:25.659697 (+ 262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:50:25.659704 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:50:25.660506 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:50:25.660507 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:50:25.661299 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:50:25.661301 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:50:25.661744 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:50:25.661745 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:50:25.661839 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:50:25.662263 (+ 424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:50:25.662271 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:50:25.663542 (+ 1271us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:50:25.665256 (+ 1714us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:50:25.665262 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:50:25.665264 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:50:25.665468 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:50:25.665663 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:50:25.665664 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:50:25.665664 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:50:25.665757 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:50:25.666007 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:50:25.666010 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:50:25.666112 (+ 102us) client_negotiation.cc:769] Sending connection context
0807 08:50:25.666189 (+ 77us) client_negotiation.cc:240] Negotiation successful
0807 08:50:25.666295 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":81,"threads_started":1}
I20260807 08:50:25.666955 28282 heartbeater.cc:347] Connected to a master server at 127.4.159.190:42791
I20260807 08:50:25.667047 28282 heartbeater.cc:464] Registering TS with master...
I20260807 08:50:25.667205 28282 heartbeater.cc:511] Master 127.4.159.190:42791 requested a full tablet report, sending...
I20260807 08:50:25.667728 28003 ts_manager.cc:194] Registered new tserver with Master: 0a8f9e196ee740fe9fdf8e9b935607d2 (127.4.159.129:42675)
WARNING: no policy specified for kudu/127.4.159.130@KRBTEST.COM; defaulting to no policy
I20260807 08:50:25.668601 28003 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.159.129@KRBTEST.COM'} at 127.4.159.129:44817
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.159.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.159.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.130.keytab.
Entry for principal kudu/127.4.159.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.130.keytab.
I20260807 08:50:25.682729 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.159.130:0
--local_ip_for_outbound_sockets=127.4.159.130
--webserver_interface=127.4.159.130
--webserver_port=0
--tserver_master_addrs=127.4.159.190:42791
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.130.keytab
--principal=kudu/127.4.159.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:33467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:50:25.761193 28289 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:50:25.761379 28289 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:50:25.761402 28289 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:50:25.762476 28289 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:50:25.762507 28289 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:50:25.762532 28289 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.159.130
I20260807 08:50:25.763876 28289 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:33467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.130.keytab
--principal=kudu/127.4.159.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.159.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.159.130
--webserver_port=0
--tserver_master_addrs=127.4.159.190:42791
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28289
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.159.130
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:50:25.764038 28289 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:50:25.764210 28289 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:50:25.768654 28299 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:50:25.768671 28296 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:50:25.768662 28295 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:50:25.768872 28289 server_base.cc:1034] running on GCE node
I20260807 08:50:25.769039 28289 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:50:25.769243 28289 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:50:25.770386 28289 hybrid_clock.cc:648] HybridClock initialized: now 1786092625770369 us; error 29 us; skew 500 ppm
Aug 07 08:50:25 dist-test-slave-6zbq krb5kdc[27941](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092625, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:50:25.772612 28289 init.cc:377] Logged in from keytab as kudu/127.4.159.130@KRBTEST.COM (short username kudu)
I20260807 08:50:25.773022 28289 webserver.cc:460] Webserver started at http://127.4.159.130:45979/ using document root <none> and password file <none>
I20260807 08:50:25.773217 28289 fs_manager.cc:359] Metadata directory not provided
I20260807 08:50:25.773263 28289 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:50:25.773384 28289 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:50:25.774322 28289 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-1/data/instance:
uuid: "bf081cf53efe44d5b6770c9d2fbf017b"
format_stamp: "Formatted at 2026-08-07 08:50:25 on dist-test-slave-6zbq"
server_key: "0bb9e0da7bc822a4aed2edf933475ba7"
server_key_iv: "24a289caf7b807a8e7434edbe7a6aaf6"
server_key_version: "encryptionkey@0"
I20260807 08:50:25.774696 28289 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-1/wal/instance:
uuid: "bf081cf53efe44d5b6770c9d2fbf017b"
format_stamp: "Formatted at 2026-08-07 08:50:25 on dist-test-slave-6zbq"
server_key: "0bb9e0da7bc822a4aed2edf933475ba7"
server_key_iv: "24a289caf7b807a8e7434edbe7a6aaf6"
server_key_version: "encryptionkey@0"
I20260807 08:50:25.775905 28289 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:50:25.776554 28309 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:50:25.776726 28289 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 08:50:25.776806 28289 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-1/wal
uuid: "bf081cf53efe44d5b6770c9d2fbf017b"
format_stamp: "Formatted at 2026-08-07 08:50:25 on dist-test-slave-6zbq"
server_key: "0bb9e0da7bc822a4aed2edf933475ba7"
server_key_iv: "24a289caf7b807a8e7434edbe7a6aaf6"
server_key_version: "encryptionkey@0"
I20260807 08:50:25.776875 28289 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:50:25.793113 28289 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:50:25.794838 28289 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:50:25.794961 28289 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:50:25.795168 28289 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:50:25.795452 28289 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:50:25.795486 28289 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:50:25.795513 28289 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:50:25.795537 28289 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:50:25.800875 28289 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.130:45125
I20260807 08:50:25.800930 28473 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.130:45125 every 8 connection(s)
I20260807 08:50:25.801215 28289 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-1/data/info.pb
Aug 07 08:50:25 dist-test-slave-6zbq krb5kdc[27941](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092625, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.130@KRBTEST.COM for kudu/127.4.159.190@KRBTEST.COM
I20260807 08:50:25.807235 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 28289
I20260807 08:50:25.807355 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-1/wal/instance
I20260807 08:50:25.807543 4734 external_mini_cluster.cc:1428] Setting key 2193caf051e2088e84f8c7d3196d718d
I20260807 08:50:25.809803 28057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:25.801814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.130:55285 (local address 127.4.159.190:42791)
0807 08:50:25.801862 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0807 08:50:25.801865 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:50:25.802054 (+ 189us) server_negotiation.cc:407] Connection header received
0807 08:50:25.802553 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:50:25.802555 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:50:25.802572 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:50:25.802628 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:50:25.803521 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:25.804009 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:25.804945 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:25.805139 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:25.805195 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:50:25.807167 (+ 1972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:50:25.807168 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:50:25.807169 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:50:25.807176 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:50:25.808533 (+ 1357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:50:25.808975 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:50:25.808976 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:50:25.808976 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:50:25.809026 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:50:25.809264 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:50:25.809265 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:50:25.809265 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:50:25.809360 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:50:25.809405 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0807 08:50:25.809672 (+ 267us) server_negotiation.cc:299] Negotiation successful
0807 08:50:25.809734 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 08:50:25.810129 28478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:25.801784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:42791 (local address 127.4.159.130:55285)
0807 08:50:25.801942 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:25.801975 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0807 08:50:25.802454 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:50:25.802698 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:50:25.802702 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:50:25.803244 (+ 542us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:50:25.803419 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:50:25.803423 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:50:25.804090 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:50:25.804092 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:50:25.804860 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:50:25.804862 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:50:25.805206 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:50:25.805207 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:50:25.805327 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:50:25.805747 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:50:25.805756 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:50:25.807075 (+ 1319us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:50:25.808601 (+ 1526us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:50:25.808609 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:50:25.808611 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:50:25.808911 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:50:25.809082 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:50:25.809083 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:50:25.809083 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:50:25.809211 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:50:25.809408 (+ 197us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:50:25.809413 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:50:25.809557 (+ 144us) client_negotiation.cc:769] Sending connection context
0807 08:50:25.809661 (+ 104us) client_negotiation.cc:240] Negotiation successful
0807 08:50:25.809812 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":77,"threads_started":1}
I20260807 08:50:25.810571 28474 heartbeater.cc:347] Connected to a master server at 127.4.159.190:42791
I20260807 08:50:25.810658 28474 heartbeater.cc:464] Registering TS with master...
I20260807 08:50:25.810822 28474 heartbeater.cc:511] Master 127.4.159.190:42791 requested a full tablet report, sending...
I20260807 08:50:25.811178 28003 ts_manager.cc:194] Registered new tserver with Master: bf081cf53efe44d5b6770c9d2fbf017b (127.4.159.130:45125)
I20260807 08:50:25.811527 28003 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.159.130@KRBTEST.COM'} at 127.4.159.130:55285
WARNING: no policy specified for kudu/127.4.159.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.159.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.159.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.131.keytab.
Entry for principal kudu/127.4.159.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.131.keytab.
I20260807 08:50:25.831096 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.159.131:0
--local_ip_for_outbound_sockets=127.4.159.131
--webserver_interface=127.4.159.131
--webserver_port=0
--tserver_master_addrs=127.4.159.190:42791
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.131.keytab
--principal=kudu/127.4.159.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:33467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:50:25.910374 28482 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:50:25.910559 28482 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:50:25.910581 28482 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:50:25.911659 28482 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:50:25.911690 28482 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:50:25.911715 28482 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.159.131
I20260807 08:50:25.913065 28482 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:33467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.131.keytab
--principal=kudu/127.4.159.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.159.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.159.131
--webserver_port=0
--tserver_master_addrs=127.4.159.190:42791
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28482
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.159.131
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:50:25.913255 28482 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:50:25.913420 28482 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:50:25.917898 28497 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:50:25.918032 28482 server_base.cc:1034] running on GCE node
W20260807 08:50:25.918097 28491 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:50:25.917907 28494 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:50:25.918413 28482 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:50:25.918630 28482 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:50:25.919764 28482 hybrid_clock.cc:648] HybridClock initialized: now 1786092625919752 us; error 29 us; skew 500 ppm
Aug 07 08:50:25 dist-test-slave-6zbq krb5kdc[27941](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092625, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:50:25.921973 28482 init.cc:377] Logged in from keytab as kudu/127.4.159.131@KRBTEST.COM (short username kudu)
I20260807 08:50:25.922394 28482 webserver.cc:460] Webserver started at http://127.4.159.131:46805/ using document root <none> and password file <none>
I20260807 08:50:25.922580 28482 fs_manager.cc:359] Metadata directory not provided
I20260807 08:50:25.922616 28482 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:50:25.922703 28482 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:50:25.923602 28482 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-2/data/instance:
uuid: "bfbbc2d0d72b4f628ef3f2522c9c2106"
format_stamp: "Formatted at 2026-08-07 08:50:25 on dist-test-slave-6zbq"
server_key: "2444b091a0eb01fa0372129703c1dae2"
server_key_iv: "826f8fdd564a007314aa095676b3776d"
server_key_version: "encryptionkey@0"
I20260807 08:50:25.923882 28482 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-2/wal/instance:
uuid: "bfbbc2d0d72b4f628ef3f2522c9c2106"
format_stamp: "Formatted at 2026-08-07 08:50:25 on dist-test-slave-6zbq"
server_key: "2444b091a0eb01fa0372129703c1dae2"
server_key_iv: "826f8fdd564a007314aa095676b3776d"
server_key_version: "encryptionkey@0"
I20260807 08:50:25.925004 28482 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260807 08:50:25.925633 28507 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:50:25.925904 28482 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 08:50:25.925998 28482 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-2/wal
uuid: "bfbbc2d0d72b4f628ef3f2522c9c2106"
format_stamp: "Formatted at 2026-08-07 08:50:25 on dist-test-slave-6zbq"
server_key: "2444b091a0eb01fa0372129703c1dae2"
server_key_iv: "826f8fdd564a007314aa095676b3776d"
server_key_version: "encryptionkey@0"
I20260807 08:50:25.926052 28482 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:50:25.934938 28482 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:50:25.936497 28482 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:50:25.936611 28482 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:50:25.936770 28482 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:50:25.937002 28482 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:50:25.937031 28482 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:50:25.937050 28482 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:50:25.937067 28482 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:50:25.942526 28482 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.131:45693
I20260807 08:50:25.942587 28688 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.131:45693 every 8 connection(s)
I20260807 08:50:25.942835 28482 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-2/data/info.pb
I20260807 08:50:25.945226 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 28482
I20260807 08:50:25.945302 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-2/wal/instance
I20260807 08:50:25.945484 4734 external_mini_cluster.cc:1428] Setting key 0e6e9abb8ac12bd0295838bd29ebf0c8
Aug 07 08:50:25 dist-test-slave-6zbq krb5kdc[27941](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092625, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.131@KRBTEST.COM for kudu/127.4.159.190@KRBTEST.COM
I20260807 08:50:25.951737 28057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:25.943439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.131:39355 (local address 127.4.159.190:42791)
0807 08:50:25.943480 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0807 08:50:25.943482 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:50:25.943671 (+ 189us) server_negotiation.cc:407] Connection header received
0807 08:50:25.944102 (+ 431us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:50:25.944103 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:50:25.944114 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:50:25.944156 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:50:25.944982 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:25.945541 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:25.946954 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:25.947273 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:25.947345 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:50:25.949441 (+ 2096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:50:25.949443 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:50:25.949443 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:50:25.949450 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:50:25.950708 (+ 1258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:50:25.951035 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:50:25.951036 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:50:25.951036 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:50:25.951083 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:50:25.951278 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:50:25.951278 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:50:25.951279 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:50:25.951373 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:50:25.951419 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0807 08:50:25.951617 (+ 198us) server_negotiation.cc:299] Negotiation successful
0807 08:50:25.951674 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260807 08:50:25.951893 28692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:25.943405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:42791 (local address 127.4.159.131:39355)
0807 08:50:25.943555 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:25.943593 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 08:50:25.944010 (+ 417us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:50:25.944227 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:50:25.944230 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:50:25.944742 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:50:25.944916 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:50:25.944920 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:50:25.945663 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:50:25.945665 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:50:25.946868 (+ 1203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:50:25.946872 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:50:25.947357 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:50:25.947358 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:50:25.947506 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:50:25.947991 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:50:25.948000 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:50:25.949357 (+ 1357us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:50:25.950780 (+ 1423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:50:25.950785 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:50:25.950787 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:50:25.950981 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:50:25.951134 (+ 153us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:50:25.951134 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:50:25.951134 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:50:25.951232 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:50:25.951416 (+ 184us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:50:25.951419 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:50:25.951517 (+ 98us) client_negotiation.cc:769] Sending connection context
0807 08:50:25.951602 (+ 85us) client_negotiation.cc:240] Negotiation successful
0807 08:50:25.951710 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":70,"threads_started":1}
I20260807 08:50:25.952316 28689 heartbeater.cc:347] Connected to a master server at 127.4.159.190:42791
I20260807 08:50:25.952397 28689 heartbeater.cc:464] Registering TS with master...
I20260807 08:50:25.952553 28689 heartbeater.cc:511] Master 127.4.159.190:42791 requested a full tablet report, sending...
I20260807 08:50:25.952857 28003 ts_manager.cc:194] Registered new tserver with Master: bfbbc2d0d72b4f628ef3f2522c9c2106 (127.4.159.131:45693)
I20260807 08:50:25.953194 28003 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.159.131@KRBTEST.COM'} at 127.4.159.131:39355
I20260807 08:50:25.957059 4734 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 08:50:25.963485 28057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:25.958261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54698 (local address 127.4.159.190:42791)
0807 08:50:25.958302 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0807 08:50:25.958304 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:50:25.958418 (+ 114us) server_negotiation.cc:407] Connection header received
0807 08:50:25.958465 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:50:25.958465 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:50:25.958474 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:50:25.958518 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:50:25.959001 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:25.959452 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:25.960103 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:25.960289 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:25.960340 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:50:25.961295 (+ 955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:50:25.961296 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:50:25.961296 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:50:25.961300 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:50:25.962667 (+ 1367us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:50:25.962951 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:50:25.962952 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:50:25.962952 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:50:25.962994 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:50:25.963107 (+ 113us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:50:25.963108 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:50:25.963108 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:50:25.963232 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:50:25.963297 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0807 08:50:25.963373 (+ 76us) server_negotiation.cc:299] Negotiation successful
0807 08:50:25.963425 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 08:50:25.964792 28003 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54698:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 08:50:25.966249 28003 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:50:25.972296 28713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:25.968511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36192 (local address 127.4.159.131:45693)
0807 08:50:25.968661 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0807 08:50:25.968666 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:50:25.968872 (+ 206us) server_negotiation.cc:407] Connection header received
0807 08:50:25.968940 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:50:25.968942 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:50:25.968956 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:50:25.969036 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:50:25.969222 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:25.970250 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:25.971299 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:25.971918 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:25.971990 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:50:25.972040 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0807 08:50:25.972185 (+ 145us) server_negotiation.cc:299] Negotiation successful
0807 08:50:25.972223 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":78,"threads_started":1}
I20260807 08:50:25.972296 28710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:25.968309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.131:45693 (local address 127.0.0.1:36192)
0807 08:50:25.968812 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:25.968821 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 08:50:25.968867 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:50:25.969095 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:50:25.969096 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:50:25.969097 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:50:25.969140 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:50:25.969142 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:50:25.970370 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:50:25.970371 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:50:25.971230 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:50:25.971233 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:50:25.972008 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:50:25.972009 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:50:25.972086 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:50:25.972098 (+ 12us) client_negotiation.cc:769] Sending connection context
0807 08:50:25.972166 (+ 68us) client_negotiation.cc:240] Negotiation successful
0807 08:50:25.972227 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":78,"threads_started":1}
I20260807 08:50:25.972469 28711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:25.968396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59252 (local address 127.4.159.129:42675)
0807 08:50:25.968692 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0807 08:50:25.968696 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:50:25.968706 (+ 10us) server_negotiation.cc:407] Connection header received
0807 08:50:25.968790 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:50:25.968793 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:50:25.968809 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:50:25.968884 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:50:25.969075 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:25.970232 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:25.971513 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:25.972004 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:25.972070 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:50:25.972126 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 08:50:25.972350 (+ 224us) server_negotiation.cc:299] Negotiation successful
0807 08:50:25.972391 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262,"thread_start_us":92,"threads_started":1}
I20260807 08:50:25.972551 28706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:25.968310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.130:45125 (local address 127.0.0.1:39960)
0807 08:50:25.968945 (+ 635us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:25.968953 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0807 08:50:25.968993 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:50:25.969288 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:50:25.969289 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:50:25.969291 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:50:25.969332 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:50:25.969333 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:50:25.970148 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:50:25.970150 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:50:25.971379 (+ 1229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:50:25.971382 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:50:25.972327 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:50:25.972328 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:50:25.972390 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:50:25.972395 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 08:50:25.972445 (+ 50us) client_negotiation.cc:240] Negotiation successful
0807 08:50:25.972485 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":626,"spinlock_wait_cycles":384,"thread_start_us":80,"threads_started":1}
I20260807 08:50:25.972684 28709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:25.968301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.129:42675 (local address 127.0.0.1:59252)
0807 08:50:25.968591 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:25.968631 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 08:50:25.968697 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:50:25.968915 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:50:25.968919 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:50:25.968921 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:50:25.968986 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:50:25.968988 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:50:25.970334 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:50:25.970334 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:50:25.971428 (+ 1094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:50:25.971430 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:50:25.972159 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:50:25.972160 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:50:25.972225 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:50:25.972229 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 08:50:25.972274 (+ 45us) client_negotiation.cc:240] Negotiation successful
0807 08:50:25.972304 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"mutex_wait_us":2,"thread_start_us":63,"threads_started":1}
I20260807 08:50:25.972998 28590 tablet_service.cc:1467] Processing CreateTablet for tablet 9f2e4018e60d495ba9f8aeb5d906ccdf (DEFAULT_TABLE table=test-table [id=26226005c83d4ce3ac5d878bf1feab84]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:50:25.973117 28714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:25.968521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39960 (local address 127.4.159.130:45125)
0807 08:50:25.968732 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0807 08:50:25.968734 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:50:25.969057 (+ 323us) server_negotiation.cc:407] Connection header received
0807 08:50:25.969107 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:50:25.969109 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:50:25.969125 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:50:25.969191 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:50:25.969402 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:25.970062 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:25.971477 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:25.972156 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:25.972232 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:50:25.972289 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 08:50:25.972987 (+ 698us) server_negotiation.cc:299] Negotiation successful
0807 08:50:25.973038 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":73,"threads_started":1}
I20260807 08:50:25.973278 28590 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9f2e4018e60d495ba9f8aeb5d906ccdf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:50:25.973320 28184 tablet_service.cc:1467] Processing CreateTablet for tablet 9f2e4018e60d495ba9f8aeb5d906ccdf (DEFAULT_TABLE table=test-table [id=26226005c83d4ce3ac5d878bf1feab84]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:50:25.973616 28184 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9f2e4018e60d495ba9f8aeb5d906ccdf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:50:25.973774 28375 tablet_service.cc:1467] Processing CreateTablet for tablet 9f2e4018e60d495ba9f8aeb5d906ccdf (DEFAULT_TABLE table=test-table [id=26226005c83d4ce3ac5d878bf1feab84]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:50:25.974077 28375 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9f2e4018e60d495ba9f8aeb5d906ccdf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:50:25.975759 28715 tablet_bootstrap.cc:492] T 9f2e4018e60d495ba9f8aeb5d906ccdf P bfbbc2d0d72b4f628ef3f2522c9c2106: Bootstrap starting.
I20260807 08:50:25.976370 28715 tablet_bootstrap.cc:654] T 9f2e4018e60d495ba9f8aeb5d906ccdf P bfbbc2d0d72b4f628ef3f2522c9c2106: Neither blocks nor log segments found. Creating new log.
I20260807 08:50:25.976744 28716 tablet_bootstrap.cc:492] T 9f2e4018e60d495ba9f8aeb5d906ccdf P 0a8f9e196ee740fe9fdf8e9b935607d2: Bootstrap starting.
I20260807 08:50:25.976902 28715 log.cc:824] T 9f2e4018e60d495ba9f8aeb5d906ccdf P bfbbc2d0d72b4f628ef3f2522c9c2106: Log is configured to *not* fsync() on all Append() calls
I20260807 08:50:25.977342 28718 tablet_bootstrap.cc:492] T 9f2e4018e60d495ba9f8aeb5d906ccdf P bf081cf53efe44d5b6770c9d2fbf017b: Bootstrap starting.
I20260807 08:50:25.977487 28716 tablet_bootstrap.cc:654] T 9f2e4018e60d495ba9f8aeb5d906ccdf P 0a8f9e196ee740fe9fdf8e9b935607d2: Neither blocks nor log segments found. Creating new log.
I20260807 08:50:25.977902 28718 tablet_bootstrap.cc:654] T 9f2e4018e60d495ba9f8aeb5d906ccdf P bf081cf53efe44d5b6770c9d2fbf017b: Neither blocks nor log segments found. Creating new log.
I20260807 08:50:25.978025 28715 tablet_bootstrap.cc:492] T 9f2e4018e60d495ba9f8aeb5d906ccdf P bfbbc2d0d72b4f628ef3f2522c9c2106: No bootstrap required, opened a new log
I20260807 08:50:25.978104 28715 ts_tablet_manager.cc:1397] T 9f2e4018e60d495ba9f8aeb5d906ccdf P bfbbc2d0d72b4f628ef3f2522c9c2106: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 08:50:25.978101 28716 log.cc:824] T 9f2e4018e60d495ba9f8aeb5d906ccdf P 0a8f9e196ee740fe9fdf8e9b935607d2: Log is configured to *not* fsync() on all Append() calls
I20260807 08:50:25.978433 28718 log.cc:824] T 9f2e4018e60d495ba9f8aeb5d906ccdf P bf081cf53efe44d5b6770c9d2fbf017b: Log is configured to *not* fsync() on all Append() calls
I20260807 08:50:25.978837 28716 tablet_bootstrap.cc:492] T 9f2e4018e60d495ba9f8aeb5d906ccdf P 0a8f9e196ee740fe9fdf8e9b935607d2: No bootstrap required, opened a new log
I20260807 08:50:25.978904 28716 ts_tablet_manager.cc:1397] T 9f2e4018e60d495ba9f8aeb5d906ccdf P 0a8f9e196ee740fe9fdf8e9b935607d2: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 08:50:25.979152 28718 tablet_bootstrap.cc:492] T 9f2e4018e60d495ba9f8aeb5d906ccdf P bf081cf53efe44d5b6770c9d2fbf017b: No bootstrap required, opened a new log
I20260807 08:50:25.979228 28718 ts_tablet_manager.cc:1397] T 9f2e4018e60d495ba9f8aeb5d906ccdf P bf081cf53efe44d5b6770c9d2fbf017b: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260807 08:50:25.979817 28715 raft_consensus.cc:348] T 9f2e4018e60d495ba9f8aeb5d906ccdf P bfbbc2d0d72b4f628ef3f2522c9c2106 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a8f9e196ee740fe9fdf8e9b935607d2" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 42675 } } peers { permanent_uuid: "bf081cf53efe44d5b6770c9d2fbf017b" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 45125 } } peers { permanent_uuid: "bfbbc2d0d72b4f628ef3f2522c9c2106" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 45693 } }
I20260807 08:50:25.979950 28715 raft_consensus.cc:374] T 9f2e4018e60d495ba9f8aeb5d906ccdf P bfbbc2d0d72b4f628ef3f2522c9c2106 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:50:25.979982 28715 raft_consensus.cc:729] T 9f2e4018e60d495ba9f8aeb5d906ccdf P bfbbc2d0d72b4f628ef3f2522c9c2106 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfbbc2d0d72b4f628ef3f2522c9c2106, State: Initialized, Role: FOLLOWER
I20260807 08:50:25.980110 28715 consensus_queue.cc:260] T 9f2e4018e60d495ba9f8aeb5d906ccdf P bfbbc2d0d72b4f628ef3f2522c9c2106 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a8f9e196ee740fe9fdf8e9b935607d2" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 42675 } } peers { permanent_uuid: "bf081cf53efe44d5b6770c9d2fbf017b" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 45125 } } peers { permanent_uuid: "bfbbc2d0d72b4f628ef3f2522c9c2106" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 45693 } }
I20260807 08:50:25.980360 28715 ts_tablet_manager.cc:1428] T 9f2e4018e60d495ba9f8aeb5d906ccdf P bfbbc2d0d72b4f628ef3f2522c9c2106: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 08:50:25.980484 28689 heartbeater.cc:503] Master 127.4.159.190:42791 was elected leader, sending a full tablet report...
I20260807 08:50:25.980757 28716 raft_consensus.cc:348] T 9f2e4018e60d495ba9f8aeb5d906ccdf P 0a8f9e196ee740fe9fdf8e9b935607d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a8f9e196ee740fe9fdf8e9b935607d2" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 42675 } } peers { permanent_uuid: "bf081cf53efe44d5b6770c9d2fbf017b" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 45125 } } peers { permanent_uuid: "bfbbc2d0d72b4f628ef3f2522c9c2106" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 45693 } }
I20260807 08:50:25.980924 28716 raft_consensus.cc:374] T 9f2e4018e60d495ba9f8aeb5d906ccdf P 0a8f9e196ee740fe9fdf8e9b935607d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:50:25.980958 28716 raft_consensus.cc:729] T 9f2e4018e60d495ba9f8aeb5d906ccdf P 0a8f9e196ee740fe9fdf8e9b935607d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a8f9e196ee740fe9fdf8e9b935607d2, State: Initialized, Role: FOLLOWER
I20260807 08:50:25.981057 28716 consensus_queue.cc:260] T 9f2e4018e60d495ba9f8aeb5d906ccdf P 0a8f9e196ee740fe9fdf8e9b935607d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a8f9e196ee740fe9fdf8e9b935607d2" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 42675 } } peers { permanent_uuid: "bf081cf53efe44d5b6770c9d2fbf017b" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 45125 } } peers { permanent_uuid: "bfbbc2d0d72b4f628ef3f2522c9c2106" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 45693 } }
I20260807 08:50:25.981006 28718 raft_consensus.cc:348] T 9f2e4018e60d495ba9f8aeb5d906ccdf P bf081cf53efe44d5b6770c9d2fbf017b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a8f9e196ee740fe9fdf8e9b935607d2" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 42675 } } peers { permanent_uuid: "bf081cf53efe44d5b6770c9d2fbf017b" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 45125 } } peers { permanent_uuid: "bfbbc2d0d72b4f628ef3f2522c9c2106" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 45693 } }
I20260807 08:50:25.981194 28718 raft_consensus.cc:374] T 9f2e4018e60d495ba9f8aeb5d906ccdf P bf081cf53efe44d5b6770c9d2fbf017b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:50:25.981246 28718 raft_consensus.cc:729] T 9f2e4018e60d495ba9f8aeb5d906ccdf P bf081cf53efe44d5b6770c9d2fbf017b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf081cf53efe44d5b6770c9d2fbf017b, State: Initialized, Role: FOLLOWER
I20260807 08:50:25.981323 28716 ts_tablet_manager.cc:1428] T 9f2e4018e60d495ba9f8aeb5d906ccdf P 0a8f9e196ee740fe9fdf8e9b935607d2: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260807 08:50:25.981348 28718 consensus_queue.cc:260] T 9f2e4018e60d495ba9f8aeb5d906ccdf P bf081cf53efe44d5b6770c9d2fbf017b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a8f9e196ee740fe9fdf8e9b935607d2" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 42675 } } peers { permanent_uuid: "bf081cf53efe44d5b6770c9d2fbf017b" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 45125 } } peers { permanent_uuid: "bfbbc2d0d72b4f628ef3f2522c9c2106" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 45693 } }
I20260807 08:50:25.981470 28282 heartbeater.cc:503] Master 127.4.159.190:42791 was elected leader, sending a full tablet report...
I20260807 08:50:25.981604 28718 ts_tablet_manager.cc:1428] T 9f2e4018e60d495ba9f8aeb5d906ccdf P bf081cf53efe44d5b6770c9d2fbf017b: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260807 08:50:25.981643 28474 heartbeater.cc:503] Master 127.4.159.190:42791 was elected leader, sending a full tablet report...
W20260807 08:50:26.051798 28475 tablet.cc:2369] T 9f2e4018e60d495ba9f8aeb5d906ccdf P bf081cf53efe44d5b6770c9d2fbf017b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 08:50:26.075570 28723 raft_consensus.cc:482] T 9f2e4018e60d495ba9f8aeb5d906ccdf P bfbbc2d0d72b4f628ef3f2522c9c2106 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:50:26.075702 28723 raft_consensus.cc:504] T 9f2e4018e60d495ba9f8aeb5d906ccdf P bfbbc2d0d72b4f628ef3f2522c9c2106 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a8f9e196ee740fe9fdf8e9b935607d2" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 42675 } } peers { permanent_uuid: "bf081cf53efe44d5b6770c9d2fbf017b" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 45125 } } peers { permanent_uuid: "bfbbc2d0d72b4f628ef3f2522c9c2106" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 45693 } }
I20260807 08:50:26.076064 28723 leader_election.cc:288] T 9f2e4018e60d495ba9f8aeb5d906ccdf P bfbbc2d0d72b4f628ef3f2522c9c2106 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a8f9e196ee740fe9fdf8e9b935607d2 (127.4.159.129:42675), bf081cf53efe44d5b6770c9d2fbf017b (127.4.159.130:45125)
I20260807 08:50:26.079778 28692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:26.076246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.130:45125 (local address 127.4.159.131:49627)
0807 08:50:26.076305 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:26.076316 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 08:50:26.076356 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:50:26.076504 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:50:26.076505 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:50:26.076507 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:50:26.076591 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:50:26.076593 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:50:26.077489 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:50:26.077490 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:50:26.078761 (+ 1271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:50:26.078765 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:50:26.079506 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:50:26.079507 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:50:26.079615 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:50:26.079618 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 08:50:26.079671 (+ 53us) client_negotiation.cc:240] Negotiation successful
0807 08:50:26.079699 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39,"mutex_wait_us":15}
I20260807 08:50:26.079792 28711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:26.076313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.131:45145 (local address 127.4.159.129:42675)
0807 08:50:26.076358 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0807 08:50:26.076360 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:50:26.076508 (+ 148us) server_negotiation.cc:407] Connection header received
0807 08:50:26.076550 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:50:26.076552 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:50:26.076560 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:50:26.076594 (+ 34us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:50:26.076726 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:26.077417 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:26.078846 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:26.079422 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:26.079495 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:50:26.079505 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0807 08:50:26.079679 (+ 174us) server_negotiation.cc:299] Negotiation successful
0807 08:50:26.079721 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 08:50:26.079790 28714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:26.076335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.131:49627 (local address 127.4.159.130:45125)
0807 08:50:26.076375 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0807 08:50:26.076377 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:50:26.076388 (+ 11us) server_negotiation.cc:407] Connection header received
0807 08:50:26.076416 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:50:26.076417 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:50:26.076426 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:50:26.076484 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:50:26.076660 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:26.077413 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:26.078847 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:26.079441 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:26.079501 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:50:26.079508 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0807 08:50:26.079679 (+ 171us) server_negotiation.cc:299] Negotiation successful
0807 08:50:26.079724 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 08:50:26.079787 28728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:26.076259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.129:42675 (local address 127.4.159.131:45145)
0807 08:50:26.076441 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:26.076449 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0807 08:50:26.076499 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:50:26.076611 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:50:26.076611 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:50:26.076613 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:50:26.076669 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:50:26.076670 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:50:26.077484 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:50:26.077485 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:50:26.078762 (+ 1277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:50:26.078765 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:50:26.079507 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:50:26.079507 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:50:26.079597 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:50:26.079602 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 08:50:26.079652 (+ 50us) client_negotiation.cc:240] Negotiation successful
0807 08:50:26.079691 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"mutex_wait_us":57,"thread_start_us":115,"threads_started":1}
I20260807 08:50:26.080124 28412 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f2e4018e60d495ba9f8aeb5d906ccdf" candidate_uuid: "bfbbc2d0d72b4f628ef3f2522c9c2106" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf081cf53efe44d5b6770c9d2fbf017b" is_pre_election: true
I20260807 08:50:26.080173 28220 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f2e4018e60d495ba9f8aeb5d906ccdf" candidate_uuid: "bfbbc2d0d72b4f628ef3f2522c9c2106" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a8f9e196ee740fe9fdf8e9b935607d2" is_pre_election: true
I20260807 08:50:26.080291 28412 raft_consensus.cc:2454] T 9f2e4018e60d495ba9f8aeb5d906ccdf P bf081cf53efe44d5b6770c9d2fbf017b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bfbbc2d0d72b4f628ef3f2522c9c2106 in term 0.
I20260807 08:50:26.080314 28220 raft_consensus.cc:2454] T 9f2e4018e60d495ba9f8aeb5d906ccdf P 0a8f9e196ee740fe9fdf8e9b935607d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bfbbc2d0d72b4f628ef3f2522c9c2106 in term 0.
I20260807 08:50:26.080534 28514 leader_election.cc:302] T 9f2e4018e60d495ba9f8aeb5d906ccdf P bfbbc2d0d72b4f628ef3f2522c9c2106 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bf081cf53efe44d5b6770c9d2fbf017b, bfbbc2d0d72b4f628ef3f2522c9c2106; no voters:
I20260807 08:50:26.080700 28723 raft_consensus.cc:2781] T 9f2e4018e60d495ba9f8aeb5d906ccdf P bfbbc2d0d72b4f628ef3f2522c9c2106 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:50:26.080761 28723 raft_consensus.cc:482] T 9f2e4018e60d495ba9f8aeb5d906ccdf P bfbbc2d0d72b4f628ef3f2522c9c2106 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:50:26.080785 28723 raft_consensus.cc:3037] T 9f2e4018e60d495ba9f8aeb5d906ccdf P bfbbc2d0d72b4f628ef3f2522c9c2106 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:50:26.081473 28723 raft_consensus.cc:504] T 9f2e4018e60d495ba9f8aeb5d906ccdf P bfbbc2d0d72b4f628ef3f2522c9c2106 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a8f9e196ee740fe9fdf8e9b935607d2" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 42675 } } peers { permanent_uuid: "bf081cf53efe44d5b6770c9d2fbf017b" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 45125 } } peers { permanent_uuid: "bfbbc2d0d72b4f628ef3f2522c9c2106" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 45693 } }
I20260807 08:50:26.081588 28723 leader_election.cc:288] T 9f2e4018e60d495ba9f8aeb5d906ccdf P bfbbc2d0d72b4f628ef3f2522c9c2106 [CANDIDATE]: Term 1 election: Requested vote from peers 0a8f9e196ee740fe9fdf8e9b935607d2 (127.4.159.129:42675), bf081cf53efe44d5b6770c9d2fbf017b (127.4.159.130:45125)
I20260807 08:50:26.081750 28412 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f2e4018e60d495ba9f8aeb5d906ccdf" candidate_uuid: "bfbbc2d0d72b4f628ef3f2522c9c2106" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf081cf53efe44d5b6770c9d2fbf017b"
I20260807 08:50:26.081830 28412 raft_consensus.cc:3037] T 9f2e4018e60d495ba9f8aeb5d906ccdf P bf081cf53efe44d5b6770c9d2fbf017b [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:50:26.081820 28220 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f2e4018e60d495ba9f8aeb5d906ccdf" candidate_uuid: "bfbbc2d0d72b4f628ef3f2522c9c2106" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a8f9e196ee740fe9fdf8e9b935607d2"
I20260807 08:50:26.081897 28220 raft_consensus.cc:3037] T 9f2e4018e60d495ba9f8aeb5d906ccdf P 0a8f9e196ee740fe9fdf8e9b935607d2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:50:26.082581 28412 raft_consensus.cc:2454] T 9f2e4018e60d495ba9f8aeb5d906ccdf P bf081cf53efe44d5b6770c9d2fbf017b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bfbbc2d0d72b4f628ef3f2522c9c2106 in term 1.
I20260807 08:50:26.082687 28220 raft_consensus.cc:2454] T 9f2e4018e60d495ba9f8aeb5d906ccdf P 0a8f9e196ee740fe9fdf8e9b935607d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bfbbc2d0d72b4f628ef3f2522c9c2106 in term 1.
I20260807 08:50:26.082774 28514 leader_election.cc:302] T 9f2e4018e60d495ba9f8aeb5d906ccdf P bfbbc2d0d72b4f628ef3f2522c9c2106 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bf081cf53efe44d5b6770c9d2fbf017b, bfbbc2d0d72b4f628ef3f2522c9c2106; no voters:
I20260807 08:50:26.082862 28723 raft_consensus.cc:2781] T 9f2e4018e60d495ba9f8aeb5d906ccdf P bfbbc2d0d72b4f628ef3f2522c9c2106 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:50:26.083005 28723 raft_consensus.cc:686] T 9f2e4018e60d495ba9f8aeb5d906ccdf P bfbbc2d0d72b4f628ef3f2522c9c2106 [term 1 LEADER]: Becoming Leader. State: Replica: bfbbc2d0d72b4f628ef3f2522c9c2106, State: Running, Role: LEADER
I20260807 08:50:26.083099 28723 consensus_queue.cc:237] T 9f2e4018e60d495ba9f8aeb5d906ccdf P bfbbc2d0d72b4f628ef3f2522c9c2106 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a8f9e196ee740fe9fdf8e9b935607d2" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 42675 } } peers { permanent_uuid: "bf081cf53efe44d5b6770c9d2fbf017b" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 45125 } } peers { permanent_uuid: "bfbbc2d0d72b4f628ef3f2522c9c2106" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 45693 } }
I20260807 08:50:26.083945 28003 catalog_manager.cc:5495] T 9f2e4018e60d495ba9f8aeb5d906ccdf P bfbbc2d0d72b4f628ef3f2522c9c2106 reported cstate change: term changed from 0 to 1, leader changed from <none> to bfbbc2d0d72b4f628ef3f2522c9c2106 (127.4.159.131). New cstate: current_term: 1 leader_uuid: "bfbbc2d0d72b4f628ef3f2522c9c2106" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a8f9e196ee740fe9fdf8e9b935607d2" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 42675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bf081cf53efe44d5b6770c9d2fbf017b" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 45125 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bfbbc2d0d72b4f628ef3f2522c9c2106" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 45693 } health_report { overall_health: HEALTHY } } }
I20260807 08:50:26.115849 28713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:26.113701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36198 (local address 127.4.159.131:45693)
0807 08:50:26.113756 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0807 08:50:26.113757 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 08:50:26.113766 (+ 9us) server_negotiation.cc:407] Connection header received
0807 08:50:26.113798 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:50:26.113799 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:50:26.113817 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:50:26.113872 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:50:26.113979 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:26.114473 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:26.115182 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:26.115376 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:26.115429 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:50:26.115562 (+ 133us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:50:26.115655 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:50:26.115723 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 08:50:26.115760 (+ 37us) server_negotiation.cc:299] Negotiation successful
0807 08:50:26.115781 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260807 08:50:26.117415 28220 raft_consensus.cc:1261] T 9f2e4018e60d495ba9f8aeb5d906ccdf P 0a8f9e196ee740fe9fdf8e9b935607d2 [term 1 FOLLOWER]: Refusing update from remote peer bfbbc2d0d72b4f628ef3f2522c9c2106: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:50:26.117700 28412 raft_consensus.cc:1261] T 9f2e4018e60d495ba9f8aeb5d906ccdf P bf081cf53efe44d5b6770c9d2fbf017b [term 1 FOLLOWER]: Refusing update from remote peer bfbbc2d0d72b4f628ef3f2522c9c2106: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:50:26.117722 28729 consensus_queue.cc:1035] T 9f2e4018e60d495ba9f8aeb5d906ccdf P bfbbc2d0d72b4f628ef3f2522c9c2106 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a8f9e196ee740fe9fdf8e9b935607d2" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:50:26.117928 28729 consensus_queue.cc:1035] T 9f2e4018e60d495ba9f8aeb5d906ccdf P bfbbc2d0d72b4f628ef3f2522c9c2106 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf081cf53efe44d5b6770c9d2fbf017b" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 45125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:50:26.119879 28732 mvcc.cc:205] Tried to move back new op lower bound from 7315835396574625792 to 7315835396437065728. Current Snapshot: MvccSnapshot[applied={T|T < 7315835396574625792}]
I20260807 08:50:26.120239 28734 mvcc.cc:205] Tried to move back new op lower bound from 7315835396574625792 to 7315835396437065728. Current Snapshot: MvccSnapshot[applied={T|T < 7315835396574625792}]
I20260807 08:50:26.120782 28733 mvcc.cc:205] Tried to move back new op lower bound from 7315835396574625792 to 7315835396437065728. Current Snapshot: MvccSnapshot[applied={T|T < 7315835396574625792}]
I20260807 08:50:26.122253 28003 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54698:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:50:26.122328 28003 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54698:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:50:26.123003 28003 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d418769ff6aa466ea11935ea8f4077d0: Sending DeleteTablet for 3 replicas of tablet 9f2e4018e60d495ba9f8aeb5d906ccdf
I20260807 08:50:26.123298 28184 tablet_service.cc:1514] Processing DeleteTablet for tablet 9f2e4018e60d495ba9f8aeb5d906ccdf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:50:26 UTC) from {username='kudu', principal='kudu/127.4.159.190@KRBTEST.COM'} at 127.0.0.1:59252
I20260807 08:50:26.123311 28375 tablet_service.cc:1514] Processing DeleteTablet for tablet 9f2e4018e60d495ba9f8aeb5d906ccdf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:50:26 UTC) from {username='kudu', principal='kudu/127.4.159.190@KRBTEST.COM'} at 127.0.0.1:39960
I20260807 08:50:26.123322 28590 tablet_service.cc:1514] Processing DeleteTablet for tablet 9f2e4018e60d495ba9f8aeb5d906ccdf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:50:26 UTC) from {username='kudu', principal='kudu/127.4.159.190@KRBTEST.COM'} at 127.0.0.1:36192
I20260807 08:50:26.123558 28746 tablet_replica.cc:330] T 9f2e4018e60d495ba9f8aeb5d906ccdf P bfbbc2d0d72b4f628ef3f2522c9c2106: stopping tablet replica
I20260807 08:50:26.123620 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 28092
I20260807 08:50:26.123651 28746 raft_consensus.cc:2229] T 9f2e4018e60d495ba9f8aeb5d906ccdf P bfbbc2d0d72b4f628ef3f2522c9c2106 [term 1 LEADER]: Raft consensus shutting down.
I20260807 08:50:26.129626 28745 tablet_replica.cc:330] T 9f2e4018e60d495ba9f8aeb5d906ccdf P bf081cf53efe44d5b6770c9d2fbf017b: stopping tablet replica
I20260807 08:50:26.129727 28745 raft_consensus.cc:2229] T 9f2e4018e60d495ba9f8aeb5d906ccdf P bf081cf53efe44d5b6770c9d2fbf017b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:50:26.129812 28745 raft_consensus.cc:2258] T 9f2e4018e60d495ba9f8aeb5d906ccdf P bf081cf53efe44d5b6770c9d2fbf017b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:50:26.131877 28745 ts_tablet_manager.cc:1905] T 9f2e4018e60d495ba9f8aeb5d906ccdf P bf081cf53efe44d5b6770c9d2fbf017b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:50:26.134038 28745 ts_tablet_manager.cc:1918] T 9f2e4018e60d495ba9f8aeb5d906ccdf P bf081cf53efe44d5b6770c9d2fbf017b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:50:26.134100 28745 log.cc:1196] T 9f2e4018e60d495ba9f8aeb5d906ccdf P bf081cf53efe44d5b6770c9d2fbf017b: Deleting WAL directory at /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-1/wal/wals/9f2e4018e60d495ba9f8aeb5d906ccdf
I20260807 08:50:26.134320 28745 ts_tablet_manager.cc:1939] T 9f2e4018e60d495ba9f8aeb5d906ccdf P bf081cf53efe44d5b6770c9d2fbf017b: Deleting consensus metadata
I20260807 08:50:26.134681 27982 catalog_manager.cc:4841] TS bf081cf53efe44d5b6770c9d2fbf017b (127.4.159.130:45125): tablet 9f2e4018e60d495ba9f8aeb5d906ccdf (table test-table [id=26226005c83d4ce3ac5d878bf1feab84]) successfully deleted
I20260807 08:50:26.138773 28746 raft_consensus.cc:2258] T 9f2e4018e60d495ba9f8aeb5d906ccdf P bfbbc2d0d72b4f628ef3f2522c9c2106 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:50:26.139336 28746 ts_tablet_manager.cc:1905] T 9f2e4018e60d495ba9f8aeb5d906ccdf P bfbbc2d0d72b4f628ef3f2522c9c2106: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:50:26.140961 28746 ts_tablet_manager.cc:1918] T 9f2e4018e60d495ba9f8aeb5d906ccdf P bfbbc2d0d72b4f628ef3f2522c9c2106: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:50:26.141068 28746 log.cc:1196] T 9f2e4018e60d495ba9f8aeb5d906ccdf P bfbbc2d0d72b4f628ef3f2522c9c2106: Deleting WAL directory at /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-2/wal/wals/9f2e4018e60d495ba9f8aeb5d906ccdf
I20260807 08:50:26.141398 28746 ts_tablet_manager.cc:1939] T 9f2e4018e60d495ba9f8aeb5d906ccdf P bfbbc2d0d72b4f628ef3f2522c9c2106: Deleting consensus metadata
I20260807 08:50:26.141825 27983 catalog_manager.cc:4841] TS bfbbc2d0d72b4f628ef3f2522c9c2106 (127.4.159.131:45693): tablet 9f2e4018e60d495ba9f8aeb5d906ccdf (table test-table [id=26226005c83d4ce3ac5d878bf1feab84]) successfully deleted
W20260807 08:50:26.142480 27984 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.159.129:42675 (error 108)
I20260807 08:50:26.142985 28709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:26.142860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.129:42675 (local address 127.0.0.1:59268)
0807 08:50:26.142913 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:26.142951 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.129:42675: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260807 08:50:26.143008 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 28289
W20260807 08:50:26.143136 27984 catalog_manager.cc:4584] TS 0a8f9e196ee740fe9fdf8e9b935607d2 (127.4.159.129:42675): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9f2e4018e60d495ba9f8aeb5d906ccdf: Network error: Client connection negotiation failed: client connection to 127.4.159.129:42675: connect: Connection refused (error 111)
I20260807 08:50:26.170892 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 28482
I20260807 08:50:26.177853 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 27960
I20260807 08:50:46.186117 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.159.190:42791
--webserver_interface=127.4.159.190
--webserver_port=33511
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab
--principal=kudu/127.4.159.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:33467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.159.190:42791
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:50:46.266939 28748 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:50:46.267120 28748 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:50:46.267140 28748 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:50:46.268244 28748 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 08:50:46.268281 28748 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 08:50:46.268293 28748 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 08:50:46.268313 28748 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:50:46.268337 28748 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:50:46.268359 28748 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:50:46.268381 28748 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:50:46.268404 28748 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:50:46.269855 28748 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:33467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.4.159.190:42791
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab
--principal=kudu/127.4.159.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.159.190:42791
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.159.190
--webserver_port=33511
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28748
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:50:46.270078 28748 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:50:46.270283 28748 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:50:46.278739 28758 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:50:46.278790 28757 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:50:46.278811 28748 server_base.cc:1034] running on GCE node
W20260807 08:50:46.278739 28760 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:50:46.279243 28748 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:50:46.279524 28748 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:50:46.280670 28748 hybrid_clock.cc:648] HybridClock initialized: now 1786092646280653 us; error 32 us; skew 500 ppm
Aug 07 08:50:46 dist-test-slave-6zbq krb5kdc[27941](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092646, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:50:46.288360 28748 init.cc:377] Logged in from keytab as kudu/127.4.159.190@KRBTEST.COM (short username kudu)
I20260807 08:50:46.288748 28748 webserver.cc:460] Webserver started at http://127.4.159.190:33511/ using document root <none> and password file <none>
I20260807 08:50:46.288935 28748 fs_manager.cc:359] Metadata directory not provided
I20260807 08:50:46.288983 28748 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:50:46.290220 28748 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:50:46.290825 28768 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:50:46.290978 28748 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 08:50:46.291064 28748 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/master-0/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/master-0/wal
uuid: "d418769ff6aa466ea11935ea8f4077d0"
format_stamp: "Formatted at 2026-08-07 08:50:25 on dist-test-slave-6zbq"
server_key: "6cf8a20136614e6d7285cef1f0474d00"
server_key_iv: "601392ea180b0a4217ec5d7c962a222b"
server_key_version: "encryptionkey@0"
I20260807 08:50:46.291447 28748 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:50:46.312496 28748 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:50:46.314543 28748 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:50:46.314658 28748 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:50:46.318415 28748 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.190:42791
I20260807 08:50:46.318455 28855 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.190:42791 every 8 connection(s)
I20260807 08:50:46.318809 28748 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
I20260807 08:50:46.320282 28856 sys_catalog.cc:263] Verifying existing consensus state
I20260807 08:50:46.320931 28856 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d418769ff6aa466ea11935ea8f4077d0: Bootstrap starting.
I20260807 08:50:46.321508 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 28748
I20260807 08:50:46.321823 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.159.129:42675
--local_ip_for_outbound_sockets=127.4.159.129
--tserver_master_addrs=127.4.159.190:42791
--webserver_port=35191
--webserver_interface=127.4.159.129
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.129.keytab
--principal=kudu/127.4.159.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:33467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 08:50:46.329906 28856 log.cc:824] T 00000000000000000000000000000000 P d418769ff6aa466ea11935ea8f4077d0: Log is configured to *not* fsync() on all Append() calls
I20260807 08:50:46.331657 28856 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d418769ff6aa466ea11935ea8f4077d0: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 08:50:46.331914 28856 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d418769ff6aa466ea11935ea8f4077d0: Bootstrap complete.
I20260807 08:50:46.333961 28856 raft_consensus.cc:348] T 00000000000000000000000000000000 P d418769ff6aa466ea11935ea8f4077d0 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d418769ff6aa466ea11935ea8f4077d0" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 42791 } }
I20260807 08:50:46.334182 28856 raft_consensus.cc:729] T 00000000000000000000000000000000 P d418769ff6aa466ea11935ea8f4077d0 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d418769ff6aa466ea11935ea8f4077d0, State: Initialized, Role: FOLLOWER
I20260807 08:50:46.334327 28856 consensus_queue.cc:260] T 00000000000000000000000000000000 P d418769ff6aa466ea11935ea8f4077d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d418769ff6aa466ea11935ea8f4077d0" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 42791 } }
I20260807 08:50:46.334415 28856 raft_consensus.cc:388] T 00000000000000000000000000000000 P d418769ff6aa466ea11935ea8f4077d0 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:50:46.334465 28856 raft_consensus.cc:482] T 00000000000000000000000000000000 P d418769ff6aa466ea11935ea8f4077d0 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:50:46.334511 28856 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d418769ff6aa466ea11935ea8f4077d0 [term 1 FOLLOWER]: Advancing to term 2
I20260807 08:50:46.335453 28856 raft_consensus.cc:504] T 00000000000000000000000000000000 P d418769ff6aa466ea11935ea8f4077d0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d418769ff6aa466ea11935ea8f4077d0" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 42791 } }
I20260807 08:50:46.335539 28856 leader_election.cc:302] T 00000000000000000000000000000000 P d418769ff6aa466ea11935ea8f4077d0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d418769ff6aa466ea11935ea8f4077d0; no voters:
I20260807 08:50:46.335690 28856 leader_election.cc:288] T 00000000000000000000000000000000 P d418769ff6aa466ea11935ea8f4077d0 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 08:50:46.335738 28862 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d418769ff6aa466ea11935ea8f4077d0 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 08:50:46.335934 28856 sys_catalog.cc:564] T 00000000000000000000000000000000 P d418769ff6aa466ea11935ea8f4077d0 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:50:46.335958 28862 raft_consensus.cc:686] T 00000000000000000000000000000000 P d418769ff6aa466ea11935ea8f4077d0 [term 2 LEADER]: Becoming Leader. State: Replica: d418769ff6aa466ea11935ea8f4077d0, State: Running, Role: LEADER
I20260807 08:50:46.336036 28862 consensus_queue.cc:237] T 00000000000000000000000000000000 P d418769ff6aa466ea11935ea8f4077d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d418769ff6aa466ea11935ea8f4077d0" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 42791 } }
I20260807 08:50:46.336275 28862 sys_catalog.cc:455] T 00000000000000000000000000000000 P d418769ff6aa466ea11935ea8f4077d0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "d418769ff6aa466ea11935ea8f4077d0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d418769ff6aa466ea11935ea8f4077d0" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 42791 } } }
I20260807 08:50:46.336349 28862 sys_catalog.cc:458] T 00000000000000000000000000000000 P d418769ff6aa466ea11935ea8f4077d0 [sys.catalog]: This master's current role is: LEADER
I20260807 08:50:46.336459 28862 sys_catalog.cc:455] T 00000000000000000000000000000000 P d418769ff6aa466ea11935ea8f4077d0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d418769ff6aa466ea11935ea8f4077d0. Latest consensus state: current_term: 2 leader_uuid: "d418769ff6aa466ea11935ea8f4077d0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d418769ff6aa466ea11935ea8f4077d0" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 42791 } } }
I20260807 08:50:46.336516 28862 sys_catalog.cc:458] T 00000000000000000000000000000000 P d418769ff6aa466ea11935ea8f4077d0 [sys.catalog]: This master's current role is: LEADER
I20260807 08:50:46.336616 28870 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:50:46.337371 28870 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:50:46.337467 28870 catalog_manager.cc:1259] Loaded cluster ID: 4ab036fec3ac4624825127d7f6e296ab
I20260807 08:50:46.337500 28870 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:50:46.338713 28870 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:50:46.338888 28870 catalog_manager.cc:5879] T 00000000000000000000000000000000 P d418769ff6aa466ea11935ea8f4077d0: Loaded TSK: 0
I20260807 08:50:46.359300 28870 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d418769ff6aa466ea11935ea8f4077d0: Generated new TSK 1
I20260807 08:50:46.359397 28870 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 08:50:46.404240 28860 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:50:46.404429 28860 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:50:46.404461 28860 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:50:46.405553 28860 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:50:46.405592 28860 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:50:46.405627 28860 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.159.129
I20260807 08:50:46.407126 28860 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:33467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.129.keytab
--principal=kudu/127.4.159.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.159.129:42675
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.159.129
--webserver_port=35191
--tserver_master_addrs=127.4.159.190:42791
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28860
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.159.129
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:50:46.407356 28860 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:50:46.407550 28860 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:50:46.413067 28901 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:50:46.413152 28904 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:50:46.413102 28900 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:50:46.413432 28860 server_base.cc:1034] running on GCE node
I20260807 08:50:46.413614 28860 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:50:46.413834 28860 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:50:46.414978 28860 hybrid_clock.cc:648] HybridClock initialized: now 1786092646414962 us; error 31 us; skew 500 ppm
Aug 07 08:50:46 dist-test-slave-6zbq krb5kdc[27941](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092646, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:50:46.417469 28860 init.cc:377] Logged in from keytab as kudu/127.4.159.129@KRBTEST.COM (short username kudu)
I20260807 08:50:46.417968 28860 webserver.cc:460] Webserver started at http://127.4.159.129:35191/ using document root <none> and password file <none>
I20260807 08:50:46.418231 28860 fs_manager.cc:359] Metadata directory not provided
I20260807 08:50:46.418322 28860 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:50:46.420033 28860 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:50:46.420756 28914 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:50:46.420915 28860 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 08:50:46.421008 28860 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-0/wal
uuid: "0a8f9e196ee740fe9fdf8e9b935607d2"
format_stamp: "Formatted at 2026-08-07 08:50:25 on dist-test-slave-6zbq"
server_key: "682ec57daf6455a5f60b2065bed6d968"
server_key_iv: "4d2c0c2ea6783ca8e60a54dc0d34f9a9"
server_key_version: "encryptionkey@0"
I20260807 08:50:46.421432 28860 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:50:46.456255 28860 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:50:46.458616 28860 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:50:46.458763 28860 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:50:46.459020 28860 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:50:46.459501 28926 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260807 08:50:46.460909 28860 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 08:50:46.460966 28860 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260807 08:50:46.460997 28860 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 08:50:46.461813 28860 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 08:50:46.461853 28860 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260807 08:50:46.461915 28926 tablet_bootstrap.cc:492] T 9f2e4018e60d495ba9f8aeb5d906ccdf P 0a8f9e196ee740fe9fdf8e9b935607d2: Bootstrap starting.
I20260807 08:50:46.468340 28860 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.129:42675
I20260807 08:50:46.468500 29084 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.129:42675 every 8 connection(s)
I20260807 08:50:46.468732 28860 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-0/data/info.pb
I20260807 08:50:46.471488 28926 log.cc:824] T 9f2e4018e60d495ba9f8aeb5d906ccdf P 0a8f9e196ee740fe9fdf8e9b935607d2: Log is configured to *not* fsync() on all Append() calls
I20260807 08:50:46.473800 28926 tablet_bootstrap.cc:492] T 9f2e4018e60d495ba9f8aeb5d906ccdf P 0a8f9e196ee740fe9fdf8e9b935607d2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 08:50:46.474074 28926 tablet_bootstrap.cc:492] T 9f2e4018e60d495ba9f8aeb5d906ccdf P 0a8f9e196ee740fe9fdf8e9b935607d2: Bootstrap complete.
I20260807 08:50:46.474848 28926 ts_tablet_manager.cc:1397] T 9f2e4018e60d495ba9f8aeb5d906ccdf P 0a8f9e196ee740fe9fdf8e9b935607d2: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.010s
I20260807 08:50:46.475840 28926 raft_consensus.cc:348] T 9f2e4018e60d495ba9f8aeb5d906ccdf P 0a8f9e196ee740fe9fdf8e9b935607d2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a8f9e196ee740fe9fdf8e9b935607d2" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 42675 } } peers { permanent_uuid: "bf081cf53efe44d5b6770c9d2fbf017b" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 45125 } } peers { permanent_uuid: "bfbbc2d0d72b4f628ef3f2522c9c2106" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 45693 } }
I20260807 08:50:46.476056 28926 raft_consensus.cc:729] T 9f2e4018e60d495ba9f8aeb5d906ccdf P 0a8f9e196ee740fe9fdf8e9b935607d2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a8f9e196ee740fe9fdf8e9b935607d2, State: Initialized, Role: FOLLOWER
I20260807 08:50:46.476202 28926 consensus_queue.cc:260] T 9f2e4018e60d495ba9f8aeb5d906ccdf P 0a8f9e196ee740fe9fdf8e9b935607d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a8f9e196ee740fe9fdf8e9b935607d2" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 42675 } } peers { permanent_uuid: "bf081cf53efe44d5b6770c9d2fbf017b" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 45125 } } peers { permanent_uuid: "bfbbc2d0d72b4f628ef3f2522c9c2106" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 45693 } }
I20260807 08:50:46.476437 28926 ts_tablet_manager.cc:1428] T 9f2e4018e60d495ba9f8aeb5d906ccdf P 0a8f9e196ee740fe9fdf8e9b935607d2: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260807 08:50:46.477675 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 28860
I20260807 08:50:46.477953 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.159.130:45125
--local_ip_for_outbound_sockets=127.4.159.130
--tserver_master_addrs=127.4.159.190:42791
--webserver_port=45979
--webserver_interface=127.4.159.130
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.130.keytab
--principal=kudu/127.4.159.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:33467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 08:50:46 dist-test-slave-6zbq krb5kdc[27941](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092646, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.129@KRBTEST.COM for kudu/127.4.159.190@KRBTEST.COM
I20260807 08:50:46.481599 29088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:46.469395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:42791 (local address 127.4.159.129:39333)
0807 08:50:46.470758 (+ 1363us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:46.470799 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 08:50:46.471382 (+ 583us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:50:46.472050 (+ 668us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:50:46.472054 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:50:46.472358 (+ 304us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:50:46.472597 (+ 239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:50:46.472602 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:50:46.474134 (+ 1532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:50:46.474135 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:50:46.475406 (+ 1271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:50:46.475409 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:50:46.476173 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:50:46.476174 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:50:46.476325 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:50:46.476799 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:50:46.476808 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:50:46.478416 (+ 1608us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:50:46.480168 (+ 1752us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:50:46.480175 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:50:46.480176 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:50:46.480380 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:50:46.480605 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:50:46.480606 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:50:46.480606 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:50:46.480706 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:50:46.481033 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:50:46.481039 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:50:46.481172 (+ 133us) client_negotiation.cc:769] Sending connection context
0807 08:50:46.481250 (+ 78us) client_negotiation.cc:240] Negotiation successful
0807 08:50:46.481409 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1324,"thread_start_us":97,"threads_started":1}
I20260807 08:50:46.481604 29089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:46.469498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.129:39333 (local address 127.4.159.190:42791)
0807 08:50:46.471691 (+ 2193us) server_negotiation.cc:206] Beginning negotiation
0807 08:50:46.471699 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 08:50:46.471737 (+ 38us) server_negotiation.cc:407] Connection header received
0807 08:50:46.471839 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:50:46.471844 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:50:46.471882 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:50:46.472022 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:50:46.472724 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:46.474015 (+ 1291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:46.475498 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:46.476072 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:46.476593 (+ 521us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:50:46.478515 (+ 1922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:50:46.478519 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:50:46.478519 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:50:46.478531 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:50:46.480082 (+ 1551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:50:46.480462 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:50:46.480465 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:50:46.480465 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:50:46.480544 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:50:46.480769 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:50:46.480771 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:50:46.480771 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:50:46.480968 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:50:46.481037 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0807 08:50:46.481336 (+ 299us) server_negotiation.cc:299] Negotiation successful
0807 08:50:46.481453 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2137,"thread_start_us":90,"threads_started":1}
I20260807 08:50:46.482276 29085 heartbeater.cc:347] Connected to a master server at 127.4.159.190:42791
I20260807 08:50:46.482399 29085 heartbeater.cc:464] Registering TS with master...
I20260807 08:50:46.482576 29085 heartbeater.cc:511] Master 127.4.159.190:42791 requested a full tablet report, sending...
I20260807 08:50:46.483305 28789 ts_manager.cc:194] Registered new tserver with Master: 0a8f9e196ee740fe9fdf8e9b935607d2 (127.4.159.129:42675)
I20260807 08:50:46.484442 28789 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.159.129@KRBTEST.COM'} at 127.4.159.129:39333
I20260807 08:50:46.485241 29085 heartbeater.cc:503] Master 127.4.159.190:42791 was elected leader, sending a full tablet report...
Aug 07 08:50:46 dist-test-slave-6zbq krb5kdc[27941](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092646, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.190@KRBTEST.COM for kudu/127.4.159.129@KRBTEST.COM
I20260807 08:50:46.491257 29096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:46.483896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.129:42675 (local address 127.0.0.1:57850)
0807 08:50:46.484090 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:46.484111 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 08:50:46.484158 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:50:46.484360 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:50:46.484366 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:50:46.484657 (+ 291us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:50:46.484697 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:50:46.484700 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:50:46.485837 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:50:46.485838 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:50:46.486405 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:50:46.486407 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:50:46.486983 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:50:46.486983 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:50:46.487028 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:50:46.487404 (+ 376us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:50:46.487407 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:50:46.488658 (+ 1251us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:50:46.490362 (+ 1704us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:50:46.490369 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:50:46.490371 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:50:46.490545 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:50:46.490735 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:50:46.490736 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:50:46.490736 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:50:46.490781 (+ 45us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:50:46.491026 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:50:46.491029 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:50:46.491064 (+ 35us) client_negotiation.cc:769] Sending connection context
0807 08:50:46.491116 (+ 52us) client_negotiation.cc:240] Negotiation successful
0807 08:50:46.491197 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":82,"threads_started":1}
I20260807 08:50:46.491324 29098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:46.483951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57850 (local address 127.4.159.129:42675)
0807 08:50:46.484243 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0807 08:50:46.484246 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:50:46.484253 (+ 7us) server_negotiation.cc:407] Connection header received
0807 08:50:46.484274 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:50:46.484275 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:50:46.484286 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:50:46.484339 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:50:46.484771 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:46.485759 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:46.486474 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:46.486907 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:46.486978 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:50:46.488741 (+ 1763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:50:46.488750 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:50:46.488751 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:50:46.488758 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:50:46.490267 (+ 1509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:50:46.490611 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:50:46.490614 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:50:46.490614 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:50:46.490675 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:50:46.490826 (+ 151us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:50:46.490826 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:50:46.490827 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:50:46.490975 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:50:46.491046 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 08:50:46.491147 (+ 101us) server_negotiation.cc:299] Negotiation successful
0807 08:50:46.491232 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":270,"thread_start_us":55,"threads_started":1}
I20260807 08:50:46.491607 28986 tablet_service.cc:1514] Processing DeleteTablet for tablet 9f2e4018e60d495ba9f8aeb5d906ccdf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:50:26 UTC) from {username='kudu', principal='kudu/127.4.159.190@KRBTEST.COM'} at 127.0.0.1:57850
I20260807 08:50:46.492008 29099 tablet_replica.cc:330] stopping tablet replica
I20260807 08:50:46.492096 29099 raft_consensus.cc:2229] T 9f2e4018e60d495ba9f8aeb5d906ccdf P 0a8f9e196ee740fe9fdf8e9b935607d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:50:46.492156 29099 raft_consensus.cc:2258] T 9f2e4018e60d495ba9f8aeb5d906ccdf P 0a8f9e196ee740fe9fdf8e9b935607d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:50:46.492419 29099 ts_tablet_manager.cc:1905] T 9f2e4018e60d495ba9f8aeb5d906ccdf P 0a8f9e196ee740fe9fdf8e9b935607d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:50:46.494045 29099 ts_tablet_manager.cc:1918] T 9f2e4018e60d495ba9f8aeb5d906ccdf P 0a8f9e196ee740fe9fdf8e9b935607d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:50:46.494105 29099 log.cc:1196] T 9f2e4018e60d495ba9f8aeb5d906ccdf P 0a8f9e196ee740fe9fdf8e9b935607d2: Deleting WAL directory at /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-0/wal/wals/9f2e4018e60d495ba9f8aeb5d906ccdf
I20260807 08:50:46.494282 29099 ts_tablet_manager.cc:1939] T 9f2e4018e60d495ba9f8aeb5d906ccdf P 0a8f9e196ee740fe9fdf8e9b935607d2: Deleting consensus metadata
I20260807 08:50:46.494688 28773 catalog_manager.cc:4841] TS 0a8f9e196ee740fe9fdf8e9b935607d2 (127.4.159.129:42675): tablet 9f2e4018e60d495ba9f8aeb5d906ccdf (table test-table [id=26226005c83d4ce3ac5d878bf1feab84]) successfully deleted
W20260807 08:50:46.565272 29095 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:50:46.565459 29095 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:50:46.565491 29095 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:50:46.566612 29095 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:50:46.566651 29095 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:50:46.566687 29095 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.159.130
I20260807 08:50:46.568109 29095 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:33467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.130.keytab
--principal=kudu/127.4.159.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.159.130:45125
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.159.130
--webserver_port=45979
--tserver_master_addrs=127.4.159.190:42791
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29095
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.159.130
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:50:46.568351 29095 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:50:46.568547 29095 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:50:46.573002 29111 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:50:46.572993 29109 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:50:46.573192 29095 server_base.cc:1034] running on GCE node
W20260807 08:50:46.573194 29113 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:50:46.573447 29095 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:50:46.573632 29095 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:50:46.574766 29095 hybrid_clock.cc:648] HybridClock initialized: now 1786092646574744 us; error 30 us; skew 500 ppm
Aug 07 08:50:46 dist-test-slave-6zbq krb5kdc[27941](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092646, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:50:46.577018 29095 init.cc:377] Logged in from keytab as kudu/127.4.159.130@KRBTEST.COM (short username kudu)
I20260807 08:50:46.577417 29095 webserver.cc:460] Webserver started at http://127.4.159.130:45979/ using document root <none> and password file <none>
I20260807 08:50:46.577617 29095 fs_manager.cc:359] Metadata directory not provided
I20260807 08:50:46.577680 29095 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:50:46.578925 29095 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 08:50:46.579449 29126 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:50:46.579591 29095 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 08:50:46.579674 29095 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-1/wal
uuid: "bf081cf53efe44d5b6770c9d2fbf017b"
format_stamp: "Formatted at 2026-08-07 08:50:25 on dist-test-slave-6zbq"
server_key: "0bb9e0da7bc822a4aed2edf933475ba7"
server_key_iv: "24a289caf7b807a8e7434edbe7a6aaf6"
server_key_version: "encryptionkey@0"
I20260807 08:50:46.579962 29095 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:50:46.591152 29095 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:50:46.592729 29095 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:50:46.592866 29095 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:50:46.593052 29095 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:50:46.593322 29095 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:50:46.593355 29095 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:50:46.593382 29095 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:50:46.593400 29095 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:50:46.598899 29095 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.130:45125
I20260807 08:50:46.598960 29303 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.130:45125 every 8 connection(s)
I20260807 08:50:46.599261 29095 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-1/data/info.pb
I20260807 08:50:46.603333 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 29095
I20260807 08:50:46.603641 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.159.131:45693
--local_ip_for_outbound_sockets=127.4.159.131
--tserver_master_addrs=127.4.159.190:42791
--webserver_port=46805
--webserver_interface=127.4.159.131
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.131.keytab
--principal=kudu/127.4.159.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:33467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 08:50:46 dist-test-slave-6zbq krb5kdc[27941](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092646, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.130@KRBTEST.COM for kudu/127.4.159.190@KRBTEST.COM
I20260807 08:50:46.608444 29089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:46.599893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.130:57881 (local address 127.4.159.190:42791)
0807 08:50:46.599947 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0807 08:50:46.599949 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:50:46.600149 (+ 200us) server_negotiation.cc:407] Connection header received
0807 08:50:46.600599 (+ 450us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:50:46.600600 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:50:46.600611 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:50:46.600662 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:50:46.601550 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:46.602045 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:46.603009 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:46.603232 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:46.603293 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:50:46.605549 (+ 2256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:50:46.605550 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:50:46.605551 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:50:46.605558 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:50:46.606861 (+ 1303us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:50:46.607306 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:50:46.607307 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:50:46.607308 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:50:46.607406 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:50:46.607694 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:50:46.607695 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:50:46.607695 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:50:46.607850 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:50:46.607914 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 08:50:46.608260 (+ 346us) server_negotiation.cc:299] Negotiation successful
0807 08:50:46.608354 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260807 08:50:46.608927 29306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:46.599882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:42791 (local address 127.4.159.130:57881)
0807 08:50:46.600033 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:46.600071 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 08:50:46.600525 (+ 454us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:50:46.600722 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:50:46.600725 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:50:46.601295 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:50:46.601476 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:50:46.601481 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:50:46.602106 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:50:46.602107 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:50:46.602937 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:50:46.602940 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:50:46.603348 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:50:46.603349 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:50:46.603544 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:50:46.604067 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:50:46.604077 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:50:46.605455 (+ 1378us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:50:46.606935 (+ 1480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:50:46.606942 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:50:46.606944 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:50:46.607224 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:50:46.607495 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:50:46.607496 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:50:46.607497 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:50:46.607630 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:50:46.607944 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:50:46.607952 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:50:46.608103 (+ 151us) client_negotiation.cc:769] Sending connection context
0807 08:50:46.608545 (+ 442us) client_negotiation.cc:240] Negotiation successful
0807 08:50:46.608713 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":67,"threads_started":1}
I20260807 08:50:46.609769 29304 heartbeater.cc:347] Connected to a master server at 127.4.159.190:42791
I20260807 08:50:46.609864 29304 heartbeater.cc:464] Registering TS with master...
I20260807 08:50:46.610026 29304 heartbeater.cc:511] Master 127.4.159.190:42791 requested a full tablet report, sending...
I20260807 08:50:46.610406 28789 ts_manager.cc:194] Registered new tserver with Master: bf081cf53efe44d5b6770c9d2fbf017b (127.4.159.130:45125)
I20260807 08:50:46.610767 28789 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.159.130@KRBTEST.COM'} at 127.4.159.130:57881
W20260807 08:50:46.686480 29308 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:50:46.686653 29308 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:50:46.686683 29308 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:50:46.687772 29308 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:50:46.687811 29308 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:50:46.687844 29308 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.159.131
I20260807 08:50:46.689237 29308 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:33467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.131.keytab
--principal=kudu/127.4.159.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.159.131:45693
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.159.131
--webserver_port=46805
--tserver_master_addrs=127.4.159.190:42791
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29308
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.159.131
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:50:46.689457 29308 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:50:46.689649 29308 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:50:46.694320 29319 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:50:46.694397 29317 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:50:46.694460 29308 server_base.cc:1034] running on GCE node
W20260807 08:50:46.694368 29315 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:50:46.694725 29308 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:50:46.694929 29308 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:50:46.696074 29308 hybrid_clock.cc:648] HybridClock initialized: now 1786092646696061 us; error 29 us; skew 500 ppm
Aug 07 08:50:46 dist-test-slave-6zbq krb5kdc[27941](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092646, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:50:46.698320 29308 init.cc:377] Logged in from keytab as kudu/127.4.159.131@KRBTEST.COM (short username kudu)
I20260807 08:50:46.698735 29308 webserver.cc:460] Webserver started at http://127.4.159.131:46805/ using document root <none> and password file <none>
I20260807 08:50:46.698959 29308 fs_manager.cc:359] Metadata directory not provided
I20260807 08:50:46.699019 29308 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:50:46.700295 29308 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:50:46.700806 29332 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:50:46.700963 29308 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 08:50:46.701043 29308 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-2/wal
uuid: "bfbbc2d0d72b4f628ef3f2522c9c2106"
format_stamp: "Formatted at 2026-08-07 08:50:25 on dist-test-slave-6zbq"
server_key: "2444b091a0eb01fa0372129703c1dae2"
server_key_iv: "826f8fdd564a007314aa095676b3776d"
server_key_version: "encryptionkey@0"
I20260807 08:50:46.701346 29308 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:50:46.714466 29308 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:50:46.716054 29308 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:50:46.716177 29308 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:50:46.716356 29308 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:50:46.716614 29308 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:50:46.716646 29308 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:50:46.716671 29308 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:50:46.716688 29308 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:50:46.721913 29308 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.131:45693
I20260807 08:50:46.721951 29501 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.131:45693 every 8 connection(s)
I20260807 08:50:46.722247 29308 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-2/data/info.pb
Aug 07 08:50:46 dist-test-slave-6zbq krb5kdc[27941](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092646, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.131@KRBTEST.COM for kudu/127.4.159.190@KRBTEST.COM
I20260807 08:50:46.730558 29089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:46.722891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.131:38525 (local address 127.4.159.190:42791)
0807 08:50:46.722950 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0807 08:50:46.722953 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:50:46.723142 (+ 189us) server_negotiation.cc:407] Connection header received
0807 08:50:46.723610 (+ 468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:50:46.723611 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:50:46.723624 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:50:46.723676 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:50:46.724548 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:46.725037 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:46.726090 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:46.726295 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:46.726390 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:50:46.728248 (+ 1858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:50:46.728249 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:50:46.728249 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:50:46.728257 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:50:46.729361 (+ 1104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:50:46.729708 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:50:46.729709 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:50:46.729710 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:50:46.729764 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:50:46.729968 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:50:46.729968 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:50:46.729969 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:50:46.730071 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:50:46.730129 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 08:50:46.730388 (+ 259us) server_negotiation.cc:299] Negotiation successful
0807 08:50:46.730470 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260807 08:50:46.730690 29506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:46.722864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:42791 (local address 127.4.159.131:38525)
0807 08:50:46.723014 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:46.723046 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0807 08:50:46.723508 (+ 462us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:50:46.723740 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:50:46.723744 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:50:46.724274 (+ 530us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:50:46.724451 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:50:46.724456 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:50:46.725131 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:50:46.725131 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:50:46.726011 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:50:46.726014 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:50:46.726375 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:50:46.726377 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:50:46.726492 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:50:46.726884 (+ 392us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:50:46.726892 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:50:46.728157 (+ 1265us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:50:46.729434 (+ 1277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:50:46.729440 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:50:46.729442 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:50:46.729642 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:50:46.729818 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:50:46.729819 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:50:46.729819 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:50:46.729913 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:50:46.730128 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:50:46.730134 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:50:46.730253 (+ 119us) client_negotiation.cc:769] Sending connection context
0807 08:50:46.730355 (+ 102us) client_negotiation.cc:240] Negotiation successful
0807 08:50:46.730498 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":76,"threads_started":1}
I20260807 08:50:46.731081 29502 heartbeater.cc:347] Connected to a master server at 127.4.159.190:42791
I20260807 08:50:46.731163 29502 heartbeater.cc:464] Registering TS with master...
I20260807 08:50:46.731323 29502 heartbeater.cc:511] Master 127.4.159.190:42791 requested a full tablet report, sending...
I20260807 08:50:46.731657 28789 ts_manager.cc:194] Registered new tserver with Master: bfbbc2d0d72b4f628ef3f2522c9c2106 (127.4.159.131:45693)
I20260807 08:50:46.731987 28789 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.159.131@KRBTEST.COM'} at 127.4.159.131:38525
I20260807 08:50:46.732116 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 29308
I20260807 08:50:46.737699 29089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:46.732483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39184 (local address 127.4.159.190:42791)
0807 08:50:46.732519 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0807 08:50:46.732521 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:50:46.732649 (+ 128us) server_negotiation.cc:407] Connection header received
0807 08:50:46.732686 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:50:46.732687 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:50:46.732696 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:50:46.732758 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:50:46.733240 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:46.733662 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:46.734393 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:46.734565 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:46.734623 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:50:46.735619 (+ 996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:50:46.735620 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:50:46.735620 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:50:46.735626 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:50:46.736855 (+ 1229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:50:46.737188 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:50:46.737189 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:50:46.737189 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:50:46.737232 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:50:46.737371 (+ 139us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:50:46.737372 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:50:46.737372 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:50:46.737450 (+ 78us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:50:46.737555 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0807 08:50:46.737592 (+ 37us) server_negotiation.cc:299] Negotiation successful
0807 08:50:46.737643 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260807 08:50:46.738329 4734 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 08:50:46.740629 29089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:46.738658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39190 (local address 127.4.159.190:42791)
0807 08:50:46.738691 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0807 08:50:46.738693 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:50:46.738801 (+ 108us) server_negotiation.cc:407] Connection header received
0807 08:50:46.738835 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:50:46.738836 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:50:46.738862 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:50:46.738923 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:50:46.738998 (+ 75us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:46.739412 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:46.740090 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:46.740279 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:46.740337 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:50:46.740456 (+ 119us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:50:46.740491 (+ 35us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0807 08:50:46.740587 (+ 96us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:39190: token expired
Metrics: {"server-negotiator.queue_time_us":15}
W20260807 08:50:46.740669 29089 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:39190: token expired
W20260807 08:50:46.740901 28697 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.4.159.190:42791)
I20260807 08:50:46.745664 29089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:46.741102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39204 (local address 127.4.159.190:42791)
0807 08:50:46.741151 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0807 08:50:46.741152 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 08:50:46.741191 (+ 39us) server_negotiation.cc:407] Connection header received
0807 08:50:46.741234 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:50:46.741235 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:50:46.741241 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:50:46.741281 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:50:46.741653 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:46.742055 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:46.742690 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:46.742848 (+ 158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:46.742891 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:50:46.743809 (+ 918us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:50:46.743810 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:50:46.743810 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:50:46.743815 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:50:46.744895 (+ 1080us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:50:46.745183 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:50:46.745184 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:50:46.745184 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:50:46.745234 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:50:46.745362 (+ 128us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:50:46.745362 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:50:46.745362 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:50:46.745441 (+ 79us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:50:46.745497 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 08:50:46.745563 (+ 66us) server_negotiation.cc:299] Negotiation successful
0807 08:50:46.745613 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260807 08:50:46.746706 28789 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39204:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 08:50:46.748162 28789 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:50:46.750048 28986 tablet_service.cc:1467] Processing CreateTablet for tablet bac38484fd6c4ddfb722397f660a7a5f (DEFAULT_TABLE table=test-table [id=68470c9174ca428f8593da879e36ed75]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:50:46.750238 28986 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bac38484fd6c4ddfb722397f660a7a5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:50:46.751751 28926 tablet_bootstrap.cc:492] T bac38484fd6c4ddfb722397f660a7a5f P 0a8f9e196ee740fe9fdf8e9b935607d2: Bootstrap starting.
I20260807 08:50:46.752023 28926 tablet_bootstrap.cc:654] T bac38484fd6c4ddfb722397f660a7a5f P 0a8f9e196ee740fe9fdf8e9b935607d2: Neither blocks nor log segments found. Creating new log.
I20260807 08:50:46.752487 28926 tablet_bootstrap.cc:492] T bac38484fd6c4ddfb722397f660a7a5f P 0a8f9e196ee740fe9fdf8e9b935607d2: No bootstrap required, opened a new log
I20260807 08:50:46.752533 28926 ts_tablet_manager.cc:1397] T bac38484fd6c4ddfb722397f660a7a5f P 0a8f9e196ee740fe9fdf8e9b935607d2: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260807 08:50:46.752897 28926 raft_consensus.cc:348] T bac38484fd6c4ddfb722397f660a7a5f P 0a8f9e196ee740fe9fdf8e9b935607d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf081cf53efe44d5b6770c9d2fbf017b" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 45125 } } peers { permanent_uuid: "bfbbc2d0d72b4f628ef3f2522c9c2106" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 45693 } } peers { permanent_uuid: "0a8f9e196ee740fe9fdf8e9b935607d2" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 42675 } }
I20260807 08:50:46.752969 28926 raft_consensus.cc:374] T bac38484fd6c4ddfb722397f660a7a5f P 0a8f9e196ee740fe9fdf8e9b935607d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:50:46.753015 28926 raft_consensus.cc:729] T bac38484fd6c4ddfb722397f660a7a5f P 0a8f9e196ee740fe9fdf8e9b935607d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a8f9e196ee740fe9fdf8e9b935607d2, State: Initialized, Role: FOLLOWER
I20260807 08:50:46.753062 28926 consensus_queue.cc:260] T bac38484fd6c4ddfb722397f660a7a5f P 0a8f9e196ee740fe9fdf8e9b935607d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf081cf53efe44d5b6770c9d2fbf017b" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 45125 } } peers { permanent_uuid: "bfbbc2d0d72b4f628ef3f2522c9c2106" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 45693 } } peers { permanent_uuid: "0a8f9e196ee740fe9fdf8e9b935607d2" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 42675 } }
I20260807 08:50:46.753153 28926 ts_tablet_manager.cc:1428] T bac38484fd6c4ddfb722397f660a7a5f P 0a8f9e196ee740fe9fdf8e9b935607d2: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260807 08:50:46.753857 29096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:46.749828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.131:45693 (local address 127.0.0.1:57950)
0807 08:50:46.749916 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:46.749924 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0807 08:50:46.749959 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:50:46.750438 (+ 479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:50:46.750439 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:50:46.750440 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:50:46.750474 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:50:46.750474 (+ 0us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:50:46.751683 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:50:46.751685 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:50:46.752725 (+ 1040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:50:46.752729 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:50:46.753580 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:50:46.753581 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:50:46.753658 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:50:46.753679 (+ 21us) client_negotiation.cc:769] Sending connection context
0807 08:50:46.753747 (+ 68us) client_negotiation.cc:240] Negotiation successful
0807 08:50:46.753790 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":77,"thread_start_us":51,"threads_started":1}
I20260807 08:50:46.753876 29512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:46.750074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57950 (local address 127.4.159.131:45693)
0807 08:50:46.750236 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0807 08:50:46.750240 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:50:46.750252 (+ 12us) server_negotiation.cc:407] Connection header received
0807 08:50:46.750286 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:50:46.750288 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:50:46.750302 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:50:46.750413 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:50:46.750523 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:46.751582 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:46.752809 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:46.753503 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:46.753566 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:50:46.753621 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 08:50:46.753764 (+ 143us) server_negotiation.cc:299] Negotiation successful
0807 08:50:46.753802 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":76,"threads_started":1}
I20260807 08:50:46.754106 29509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:46.749777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.130:45125 (local address 127.0.0.1:52016)
0807 08:50:46.749912 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:46.749922 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 08:50:46.749961 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:50:46.750854 (+ 893us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:50:46.750855 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:50:46.750857 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:50:46.750894 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:50:46.750895 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:50:46.752201 (+ 1306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:50:46.752202 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:50:46.753005 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:50:46.753007 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:50:46.753867 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:50:46.753868 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:50:46.753925 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:50:46.753929 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 08:50:46.754011 (+ 82us) client_negotiation.cc:240] Negotiation successful
0807 08:50:46.754046 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121}
I20260807 08:50:46.754169 29511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:46.749873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52016 (local address 127.4.159.130:45125)
0807 08:50:46.750579 (+ 706us) server_negotiation.cc:206] Beginning negotiation
0807 08:50:46.750583 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:50:46.750590 (+ 7us) server_negotiation.cc:407] Connection header received
0807 08:50:46.750629 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:50:46.750630 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:50:46.750642 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:50:46.750770 (+ 128us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:50:46.750972 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:46.751673 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:46.753076 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:46.753727 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:46.753783 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:50:46.753834 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0807 08:50:46.754037 (+ 203us) server_negotiation.cc:299] Negotiation successful
0807 08:50:46.754085 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":679,"thread_start_us":111,"threads_started":1}
I20260807 08:50:46.754490 29398 tablet_service.cc:1467] Processing CreateTablet for tablet bac38484fd6c4ddfb722397f660a7a5f (DEFAULT_TABLE table=test-table [id=68470c9174ca428f8593da879e36ed75]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:50:46.754815 29398 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bac38484fd6c4ddfb722397f660a7a5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:50:46.754908 29199 tablet_service.cc:1467] Processing CreateTablet for tablet bac38484fd6c4ddfb722397f660a7a5f (DEFAULT_TABLE table=test-table [id=68470c9174ca428f8593da879e36ed75]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:50:46.755225 29199 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bac38484fd6c4ddfb722397f660a7a5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:50:46.758077 29516 tablet_bootstrap.cc:492] T bac38484fd6c4ddfb722397f660a7a5f P bfbbc2d0d72b4f628ef3f2522c9c2106: Bootstrap starting.
I20260807 08:50:46.758385 29517 tablet_bootstrap.cc:492] T bac38484fd6c4ddfb722397f660a7a5f P bf081cf53efe44d5b6770c9d2fbf017b: Bootstrap starting.
I20260807 08:50:46.758978 29517 tablet_bootstrap.cc:654] T bac38484fd6c4ddfb722397f660a7a5f P bf081cf53efe44d5b6770c9d2fbf017b: Neither blocks nor log segments found. Creating new log.
I20260807 08:50:46.758977 29516 tablet_bootstrap.cc:654] T bac38484fd6c4ddfb722397f660a7a5f P bfbbc2d0d72b4f628ef3f2522c9c2106: Neither blocks nor log segments found. Creating new log.
I20260807 08:50:46.759567 29516 log.cc:824] T bac38484fd6c4ddfb722397f660a7a5f P bfbbc2d0d72b4f628ef3f2522c9c2106: Log is configured to *not* fsync() on all Append() calls
I20260807 08:50:46.759567 29517 log.cc:824] T bac38484fd6c4ddfb722397f660a7a5f P bf081cf53efe44d5b6770c9d2fbf017b: Log is configured to *not* fsync() on all Append() calls
I20260807 08:50:46.760222 29517 tablet_bootstrap.cc:492] T bac38484fd6c4ddfb722397f660a7a5f P bf081cf53efe44d5b6770c9d2fbf017b: No bootstrap required, opened a new log
I20260807 08:50:46.760293 29517 ts_tablet_manager.cc:1397] T bac38484fd6c4ddfb722397f660a7a5f P bf081cf53efe44d5b6770c9d2fbf017b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 08:50:46.760422 29516 tablet_bootstrap.cc:492] T bac38484fd6c4ddfb722397f660a7a5f P bfbbc2d0d72b4f628ef3f2522c9c2106: No bootstrap required, opened a new log
I20260807 08:50:46.760481 29516 ts_tablet_manager.cc:1397] T bac38484fd6c4ddfb722397f660a7a5f P bfbbc2d0d72b4f628ef3f2522c9c2106: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 08:50:46.762126 29517 raft_consensus.cc:348] T bac38484fd6c4ddfb722397f660a7a5f P bf081cf53efe44d5b6770c9d2fbf017b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf081cf53efe44d5b6770c9d2fbf017b" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 45125 } } peers { permanent_uuid: "bfbbc2d0d72b4f628ef3f2522c9c2106" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 45693 } } peers { permanent_uuid: "0a8f9e196ee740fe9fdf8e9b935607d2" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 42675 } }
I20260807 08:50:46.762300 29517 raft_consensus.cc:374] T bac38484fd6c4ddfb722397f660a7a5f P bf081cf53efe44d5b6770c9d2fbf017b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:50:46.762264 29516 raft_consensus.cc:348] T bac38484fd6c4ddfb722397f660a7a5f P bfbbc2d0d72b4f628ef3f2522c9c2106 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf081cf53efe44d5b6770c9d2fbf017b" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 45125 } } peers { permanent_uuid: "bfbbc2d0d72b4f628ef3f2522c9c2106" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 45693 } } peers { permanent_uuid: "0a8f9e196ee740fe9fdf8e9b935607d2" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 42675 } }
I20260807 08:50:46.762451 29516 raft_consensus.cc:374] T bac38484fd6c4ddfb722397f660a7a5f P bfbbc2d0d72b4f628ef3f2522c9c2106 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:50:46.762465 29517 raft_consensus.cc:729] T bac38484fd6c4ddfb722397f660a7a5f P bf081cf53efe44d5b6770c9d2fbf017b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf081cf53efe44d5b6770c9d2fbf017b, State: Initialized, Role: FOLLOWER
I20260807 08:50:46.762498 29516 raft_consensus.cc:729] T bac38484fd6c4ddfb722397f660a7a5f P bfbbc2d0d72b4f628ef3f2522c9c2106 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfbbc2d0d72b4f628ef3f2522c9c2106, State: Initialized, Role: FOLLOWER
I20260807 08:50:46.762594 29516 consensus_queue.cc:260] T bac38484fd6c4ddfb722397f660a7a5f P bfbbc2d0d72b4f628ef3f2522c9c2106 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf081cf53efe44d5b6770c9d2fbf017b" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 45125 } } peers { permanent_uuid: "bfbbc2d0d72b4f628ef3f2522c9c2106" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 45693 } } peers { permanent_uuid: "0a8f9e196ee740fe9fdf8e9b935607d2" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 42675 } }
I20260807 08:50:46.762589 29517 consensus_queue.cc:260] T bac38484fd6c4ddfb722397f660a7a5f P bf081cf53efe44d5b6770c9d2fbf017b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf081cf53efe44d5b6770c9d2fbf017b" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 45125 } } peers { permanent_uuid: "bfbbc2d0d72b4f628ef3f2522c9c2106" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 45693 } } peers { permanent_uuid: "0a8f9e196ee740fe9fdf8e9b935607d2" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 42675 } }
I20260807 08:50:46.762882 29502 heartbeater.cc:503] Master 127.4.159.190:42791 was elected leader, sending a full tablet report...
I20260807 08:50:46.762882 29516 ts_tablet_manager.cc:1428] T bac38484fd6c4ddfb722397f660a7a5f P bfbbc2d0d72b4f628ef3f2522c9c2106: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260807 08:50:46.762915 29304 heartbeater.cc:503] Master 127.4.159.190:42791 was elected leader, sending a full tablet report...
I20260807 08:50:46.763163 29517 ts_tablet_manager.cc:1428] T bac38484fd6c4ddfb722397f660a7a5f P bf081cf53efe44d5b6770c9d2fbf017b: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260807 08:50:46.803922 29527 raft_consensus.cc:482] T bac38484fd6c4ddfb722397f660a7a5f P bf081cf53efe44d5b6770c9d2fbf017b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:50:46.804018 29527 raft_consensus.cc:504] T bac38484fd6c4ddfb722397f660a7a5f P bf081cf53efe44d5b6770c9d2fbf017b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf081cf53efe44d5b6770c9d2fbf017b" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 45125 } } peers { permanent_uuid: "bfbbc2d0d72b4f628ef3f2522c9c2106" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 45693 } } peers { permanent_uuid: "0a8f9e196ee740fe9fdf8e9b935607d2" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 42675 } }
I20260807 08:50:46.804311 29527 leader_election.cc:288] T bac38484fd6c4ddfb722397f660a7a5f P bf081cf53efe44d5b6770c9d2fbf017b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bfbbc2d0d72b4f628ef3f2522c9c2106 (127.4.159.131:45693), 0a8f9e196ee740fe9fdf8e9b935607d2 (127.4.159.129:42675)
I20260807 08:50:46.808005 29098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:46.804560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.130:45521 (local address 127.4.159.129:42675)
0807 08:50:46.804600 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0807 08:50:46.804601 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 08:50:46.804608 (+ 7us) server_negotiation.cc:407] Connection header received
0807 08:50:46.804654 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:50:46.804655 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:50:46.804674 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:50:46.804731 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:50:46.804885 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:46.805663 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:46.807075 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:46.807679 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:46.807739 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:50:46.807804 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0807 08:50:46.807898 (+ 94us) server_negotiation.cc:299] Negotiation successful
0807 08:50:46.807938 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 08:50:46.808004 29512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:46.804511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.130:58347 (local address 127.4.159.131:45693)
0807 08:50:46.804585 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0807 08:50:46.804588 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:50:46.804727 (+ 139us) server_negotiation.cc:407] Connection header received
0807 08:50:46.804788 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:50:46.804788 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:50:46.804795 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:50:46.804835 (+ 40us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:50:46.804974 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:46.805668 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:46.807095 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:46.807650 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:46.807726 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:50:46.807736 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0807 08:50:46.807899 (+ 163us) server_negotiation.cc:299] Negotiation successful
0807 08:50:46.807935 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 08:50:46.808028 29306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:46.804469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.129:42675 (local address 127.4.159.130:45521)
0807 08:50:46.804520 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:46.804531 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 08:50:46.804572 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:50:46.804748 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:50:46.804749 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:50:46.804751 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:50:46.804822 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:50:46.804824 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:50:46.805745 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:50:46.805746 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:50:46.806997 (+ 1251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:50:46.807000 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:50:46.807741 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:50:46.807742 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:50:46.807834 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:50:46.807835 (+ 1us) client_negotiation.cc:769] Sending connection context
0807 08:50:46.807886 (+ 51us) client_negotiation.cc:240] Negotiation successful
0807 08:50:46.807937 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34,"mutex_wait_us":36}
I20260807 08:50:46.808035 29529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:46.804471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.131:45693 (local address 127.4.159.130:58347)
0807 08:50:46.804667 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:46.804676 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 08:50:46.804724 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:50:46.804862 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:50:46.804862 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:50:46.804864 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:50:46.804917 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:50:46.804918 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:50:46.805745 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:50:46.805755 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:50:46.806997 (+ 1242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:50:46.807000 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:50:46.807725 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:50:46.807726 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:50:46.807830 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:50:46.807835 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 08:50:46.807886 (+ 51us) client_negotiation.cc:240] Negotiation successful
0807 08:50:46.807937 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"mutex_wait_us":58,"thread_start_us":96,"threads_started":1}
I20260807 08:50:46.808322 29019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bac38484fd6c4ddfb722397f660a7a5f" candidate_uuid: "bf081cf53efe44d5b6770c9d2fbf017b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a8f9e196ee740fe9fdf8e9b935607d2" is_pre_election: true
I20260807 08:50:46.808454 29019 raft_consensus.cc:2454] T bac38484fd6c4ddfb722397f660a7a5f P 0a8f9e196ee740fe9fdf8e9b935607d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf081cf53efe44d5b6770c9d2fbf017b in term 0.
I20260807 08:50:46.808518 29425 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bac38484fd6c4ddfb722397f660a7a5f" candidate_uuid: "bf081cf53efe44d5b6770c9d2fbf017b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfbbc2d0d72b4f628ef3f2522c9c2106" is_pre_election: true
I20260807 08:50:46.808651 29425 raft_consensus.cc:2454] T bac38484fd6c4ddfb722397f660a7a5f P bfbbc2d0d72b4f628ef3f2522c9c2106 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf081cf53efe44d5b6770c9d2fbf017b in term 0.
I20260807 08:50:46.808677 29131 leader_election.cc:302] T bac38484fd6c4ddfb722397f660a7a5f P bf081cf53efe44d5b6770c9d2fbf017b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a8f9e196ee740fe9fdf8e9b935607d2, bf081cf53efe44d5b6770c9d2fbf017b; no voters:
I20260807 08:50:46.808851 29527 raft_consensus.cc:2781] T bac38484fd6c4ddfb722397f660a7a5f P bf081cf53efe44d5b6770c9d2fbf017b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:50:46.808914 29527 raft_consensus.cc:482] T bac38484fd6c4ddfb722397f660a7a5f P bf081cf53efe44d5b6770c9d2fbf017b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:50:46.808936 29527 raft_consensus.cc:3037] T bac38484fd6c4ddfb722397f660a7a5f P bf081cf53efe44d5b6770c9d2fbf017b [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:50:46.809566 29527 raft_consensus.cc:504] T bac38484fd6c4ddfb722397f660a7a5f P bf081cf53efe44d5b6770c9d2fbf017b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf081cf53efe44d5b6770c9d2fbf017b" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 45125 } } peers { permanent_uuid: "bfbbc2d0d72b4f628ef3f2522c9c2106" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 45693 } } peers { permanent_uuid: "0a8f9e196ee740fe9fdf8e9b935607d2" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 42675 } }
I20260807 08:50:46.809672 29527 leader_election.cc:288] T bac38484fd6c4ddfb722397f660a7a5f P bf081cf53efe44d5b6770c9d2fbf017b [CANDIDATE]: Term 1 election: Requested vote from peers bfbbc2d0d72b4f628ef3f2522c9c2106 (127.4.159.131:45693), 0a8f9e196ee740fe9fdf8e9b935607d2 (127.4.159.129:42675)
I20260807 08:50:46.809821 29425 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bac38484fd6c4ddfb722397f660a7a5f" candidate_uuid: "bf081cf53efe44d5b6770c9d2fbf017b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfbbc2d0d72b4f628ef3f2522c9c2106"
I20260807 08:50:46.809883 29019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bac38484fd6c4ddfb722397f660a7a5f" candidate_uuid: "bf081cf53efe44d5b6770c9d2fbf017b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a8f9e196ee740fe9fdf8e9b935607d2"
I20260807 08:50:46.809900 29425 raft_consensus.cc:3037] T bac38484fd6c4ddfb722397f660a7a5f P bfbbc2d0d72b4f628ef3f2522c9c2106 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:50:46.809957 29019 raft_consensus.cc:3037] T bac38484fd6c4ddfb722397f660a7a5f P 0a8f9e196ee740fe9fdf8e9b935607d2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:50:46.810689 29425 raft_consensus.cc:2454] T bac38484fd6c4ddfb722397f660a7a5f P bfbbc2d0d72b4f628ef3f2522c9c2106 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf081cf53efe44d5b6770c9d2fbf017b in term 1.
I20260807 08:50:46.810694 29019 raft_consensus.cc:2454] T bac38484fd6c4ddfb722397f660a7a5f P 0a8f9e196ee740fe9fdf8e9b935607d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf081cf53efe44d5b6770c9d2fbf017b in term 1.
I20260807 08:50:46.810842 29131 leader_election.cc:302] T bac38484fd6c4ddfb722397f660a7a5f P bf081cf53efe44d5b6770c9d2fbf017b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a8f9e196ee740fe9fdf8e9b935607d2, bf081cf53efe44d5b6770c9d2fbf017b; no voters:
I20260807 08:50:46.810951 29527 raft_consensus.cc:2781] T bac38484fd6c4ddfb722397f660a7a5f P bf081cf53efe44d5b6770c9d2fbf017b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:50:46.811089 29527 raft_consensus.cc:686] T bac38484fd6c4ddfb722397f660a7a5f P bf081cf53efe44d5b6770c9d2fbf017b [term 1 LEADER]: Becoming Leader. State: Replica: bf081cf53efe44d5b6770c9d2fbf017b, State: Running, Role: LEADER
I20260807 08:50:46.811179 29527 consensus_queue.cc:237] T bac38484fd6c4ddfb722397f660a7a5f P bf081cf53efe44d5b6770c9d2fbf017b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf081cf53efe44d5b6770c9d2fbf017b" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 45125 } } peers { permanent_uuid: "bfbbc2d0d72b4f628ef3f2522c9c2106" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 45693 } } peers { permanent_uuid: "0a8f9e196ee740fe9fdf8e9b935607d2" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 42675 } }
I20260807 08:50:46.811936 28789 catalog_manager.cc:5495] T bac38484fd6c4ddfb722397f660a7a5f P bf081cf53efe44d5b6770c9d2fbf017b reported cstate change: term changed from 0 to 1, leader changed from <none> to bf081cf53efe44d5b6770c9d2fbf017b (127.4.159.130). New cstate: current_term: 1 leader_uuid: "bf081cf53efe44d5b6770c9d2fbf017b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf081cf53efe44d5b6770c9d2fbf017b" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 45125 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bfbbc2d0d72b4f628ef3f2522c9c2106" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 45693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a8f9e196ee740fe9fdf8e9b935607d2" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 42675 } health_report { overall_health: UNKNOWN } } }
I20260807 08:50:46.828961 29511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:46.826937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52024 (local address 127.4.159.130:45125)
0807 08:50:46.826976 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0807 08:50:46.826978 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:50:46.826986 (+ 8us) server_negotiation.cc:407] Connection header received
0807 08:50:46.827044 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:50:46.827044 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:50:46.827060 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:50:46.827113 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:50:46.827223 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:46.827634 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:46.828314 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:50:46.828488 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:50:46.828538 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:50:46.828674 (+ 136us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:50:46.828762 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:50:46.828827 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0807 08:50:46.828864 (+ 37us) server_negotiation.cc:299] Negotiation successful
0807 08:50:46.828882 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 08:50:46.830461 29019 raft_consensus.cc:1261] T bac38484fd6c4ddfb722397f660a7a5f P 0a8f9e196ee740fe9fdf8e9b935607d2 [term 1 FOLLOWER]: Refusing update from remote peer bf081cf53efe44d5b6770c9d2fbf017b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:50:46.830461 29425 raft_consensus.cc:1261] T bac38484fd6c4ddfb722397f660a7a5f P bfbbc2d0d72b4f628ef3f2522c9c2106 [term 1 FOLLOWER]: Refusing update from remote peer bf081cf53efe44d5b6770c9d2fbf017b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:50:46.830775 29530 consensus_queue.cc:1035] T bac38484fd6c4ddfb722397f660a7a5f P bf081cf53efe44d5b6770c9d2fbf017b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfbbc2d0d72b4f628ef3f2522c9c2106" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 45693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:50:46.830895 29527 consensus_queue.cc:1035] T bac38484fd6c4ddfb722397f660a7a5f P bf081cf53efe44d5b6770c9d2fbf017b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a8f9e196ee740fe9fdf8e9b935607d2" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:50:46.833261 29534 mvcc.cc:205] Tried to move back new op lower bound from 7315835481415340032 to 7315835481339195392. Current Snapshot: MvccSnapshot[applied={T|T < 7315835481415340032}]
I20260807 08:50:46.833828 29535 mvcc.cc:205] Tried to move back new op lower bound from 7315835481415340032 to 7315835481339195392. Current Snapshot: MvccSnapshot[applied={T|T < 7315835481415340032}]
I20260807 08:50:46.834870 28788 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39204:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:50:46.834962 28788 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39204:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:50:46.835610 28788 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d418769ff6aa466ea11935ea8f4077d0: Sending DeleteTablet for 3 replicas of tablet bac38484fd6c4ddfb722397f660a7a5f
I20260807 08:50:46.835901 28986 tablet_service.cc:1514] Processing DeleteTablet for tablet bac38484fd6c4ddfb722397f660a7a5f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:50:46 UTC) from {username='kudu', principal='kudu/127.4.159.190@KRBTEST.COM'} at 127.0.0.1:57850
I20260807 08:50:46.835943 29398 tablet_service.cc:1514] Processing DeleteTablet for tablet bac38484fd6c4ddfb722397f660a7a5f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:50:46 UTC) from {username='kudu', principal='kudu/127.4.159.190@KRBTEST.COM'} at 127.0.0.1:57950
I20260807 08:50:46.836011 29099 tablet_replica.cc:330] T bac38484fd6c4ddfb722397f660a7a5f P 0a8f9e196ee740fe9fdf8e9b935607d2: stopping tablet replica
I20260807 08:50:46.836082 29099 raft_consensus.cc:2229] T bac38484fd6c4ddfb722397f660a7a5f P 0a8f9e196ee740fe9fdf8e9b935607d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:50:46.836133 29099 raft_consensus.cc:2258] T bac38484fd6c4ddfb722397f660a7a5f P 0a8f9e196ee740fe9fdf8e9b935607d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:50:46.836128 29199 tablet_service.cc:1514] Processing DeleteTablet for tablet bac38484fd6c4ddfb722397f660a7a5f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:50:46 UTC) from {username='kudu', principal='kudu/127.4.159.190@KRBTEST.COM'} at 127.0.0.1:52016
I20260807 08:50:46.836328 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 28860
I20260807 08:50:46.836395 29099 ts_tablet_manager.cc:1905] T bac38484fd6c4ddfb722397f660a7a5f P 0a8f9e196ee740fe9fdf8e9b935607d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:50:46.846007 29547 tablet_replica.cc:330] T bac38484fd6c4ddfb722397f660a7a5f P bfbbc2d0d72b4f628ef3f2522c9c2106: stopping tablet replica
I20260807 08:50:46.846114 29547 raft_consensus.cc:2229] T bac38484fd6c4ddfb722397f660a7a5f P bfbbc2d0d72b4f628ef3f2522c9c2106 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:50:46.846227 29547 raft_consensus.cc:2258] T bac38484fd6c4ddfb722397f660a7a5f P bfbbc2d0d72b4f628ef3f2522c9c2106 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:50:46.850715 29547 ts_tablet_manager.cc:1905] T bac38484fd6c4ddfb722397f660a7a5f P bfbbc2d0d72b4f628ef3f2522c9c2106: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260807 08:50:46.852028 28773 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.159.129:42675 (error 108)
I20260807 08:50:46.852226 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 29095
I20260807 08:50:46.855893 29509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:46.855792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.129:42675 (local address 127.0.0.1:57864)
0807 08:50:46.855835 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:46.855863 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.129:42675: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260807 08:50:46.856063 28773 catalog_manager.cc:4584] TS 0a8f9e196ee740fe9fdf8e9b935607d2 (127.4.159.129:42675): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet bac38484fd6c4ddfb722397f660a7a5f: Network error: Client connection negotiation failed: client connection to 127.4.159.129:42675: connect: Connection refused (error 111)
I20260807 08:50:46.852283 29547 ts_tablet_manager.cc:1918] T bac38484fd6c4ddfb722397f660a7a5f P bfbbc2d0d72b4f628ef3f2522c9c2106: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:50:46.857892 29547 log.cc:1196] T bac38484fd6c4ddfb722397f660a7a5f P bfbbc2d0d72b4f628ef3f2522c9c2106: Deleting WAL directory at /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-2/wal/wals/bac38484fd6c4ddfb722397f660a7a5f
I20260807 08:50:46.858129 29547 ts_tablet_manager.cc:1939] T bac38484fd6c4ddfb722397f660a7a5f P bfbbc2d0d72b4f628ef3f2522c9c2106: Deleting consensus metadata
I20260807 08:50:46.858848 28772 catalog_manager.cc:4841] TS bfbbc2d0d72b4f628ef3f2522c9c2106 (127.4.159.131:45693): tablet bac38484fd6c4ddfb722397f660a7a5f (table test-table [id=68470c9174ca428f8593da879e36ed75]) successfully deleted
I20260807 08:50:46.864344 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 29308
I20260807 08:50:46.864365 29509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:50:46.864246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.130:45125 (local address 127.0.0.1:52034)
0807 08:50:46.864312 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0807 08:50:46.864340 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.130:45125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260807 08:50:46.874533 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 28748
I20260807 08:51:06.882903 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.159.190:42791
--webserver_interface=127.4.159.190
--webserver_port=33511
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab
--principal=kudu/127.4.159.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:33467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.159.190:42791
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:51:06.962585 29552 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:51:06.962771 29552 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:51:06.962800 29552 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:51:06.963902 29552 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 08:51:06.963940 29552 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 08:51:06.963956 29552 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 08:51:06.963971 29552 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:51:06.963989 29552 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:51:06.964012 29552 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:51:06.964030 29552 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:51:06.964054 29552 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:51:06.965492 29552 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:33467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.4.159.190:42791
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab
--principal=kudu/127.4.159.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.159.190:42791
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.159.190
--webserver_port=33511
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29552
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:51:06.965718 29552 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:51:06.965914 29552 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:51:06.970276 29561 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:51:06.970300 29560 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:51:06.970320 29564 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:51:06.970475 29552 server_base.cc:1034] running on GCE node
I20260807 08:51:06.970678 29552 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:51:06.970896 29552 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:51:06.972035 29552 hybrid_clock.cc:648] HybridClock initialized: now 1786092666972023 us; error 27 us; skew 500 ppm
Aug 07 08:51:06 dist-test-slave-6zbq krb5kdc[27941](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092666, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:51:06.977279 29552 init.cc:377] Logged in from keytab as kudu/127.4.159.190@KRBTEST.COM (short username kudu)
I20260807 08:51:06.977672 29552 webserver.cc:460] Webserver started at http://127.4.159.190:33511/ using document root <none> and password file <none>
I20260807 08:51:06.977874 29552 fs_manager.cc:359] Metadata directory not provided
I20260807 08:51:06.977916 29552 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:51:06.979111 29552 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.000s
I20260807 08:51:06.979696 29581 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:51:06.979851 29552 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 08:51:06.979929 29552 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/master-0/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/master-0/wal
uuid: "d418769ff6aa466ea11935ea8f4077d0"
format_stamp: "Formatted at 2026-08-07 08:50:25 on dist-test-slave-6zbq"
server_key: "6cf8a20136614e6d7285cef1f0474d00"
server_key_iv: "601392ea180b0a4217ec5d7c962a222b"
server_key_version: "encryptionkey@0"
I20260807 08:51:06.980212 29552 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:51:06.988080 29552 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:51:06.989923 29552 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:51:06.990022 29552 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:51:06.993616 29552 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.190:42791
I20260807 08:51:06.993646 29644 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.190:42791 every 8 connection(s)
I20260807 08:51:06.993958 29552 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
I20260807 08:51:06.995280 29647 sys_catalog.cc:263] Verifying existing consensus state
I20260807 08:51:06.995884 29647 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d418769ff6aa466ea11935ea8f4077d0: Bootstrap starting.
I20260807 08:51:06.998116 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 29552
I20260807 08:51:06.998397 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.159.129:42675
--local_ip_for_outbound_sockets=127.4.159.129
--tserver_master_addrs=127.4.159.190:42791
--webserver_port=35191
--webserver_interface=127.4.159.129
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.129.keytab
--principal=kudu/127.4.159.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:33467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 08:51:07.004655 29647 log.cc:824] T 00000000000000000000000000000000 P d418769ff6aa466ea11935ea8f4077d0: Log is configured to *not* fsync() on all Append() calls
I20260807 08:51:07.006763 29647 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d418769ff6aa466ea11935ea8f4077d0: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 08:51:07.007006 29647 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d418769ff6aa466ea11935ea8f4077d0: Bootstrap complete.
I20260807 08:51:07.009012 29647 raft_consensus.cc:348] T 00000000000000000000000000000000 P d418769ff6aa466ea11935ea8f4077d0 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d418769ff6aa466ea11935ea8f4077d0" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 42791 } }
I20260807 08:51:07.009222 29647 raft_consensus.cc:729] T 00000000000000000000000000000000 P d418769ff6aa466ea11935ea8f4077d0 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: d418769ff6aa466ea11935ea8f4077d0, State: Initialized, Role: FOLLOWER
I20260807 08:51:07.009362 29647 consensus_queue.cc:260] T 00000000000000000000000000000000 P d418769ff6aa466ea11935ea8f4077d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d418769ff6aa466ea11935ea8f4077d0" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 42791 } }
I20260807 08:51:07.009435 29647 raft_consensus.cc:388] T 00000000000000000000000000000000 P d418769ff6aa466ea11935ea8f4077d0 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:51:07.009482 29647 raft_consensus.cc:482] T 00000000000000000000000000000000 P d418769ff6aa466ea11935ea8f4077d0 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:51:07.009527 29647 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d418769ff6aa466ea11935ea8f4077d0 [term 2 FOLLOWER]: Advancing to term 3
I20260807 08:51:07.010474 29647 raft_consensus.cc:504] T 00000000000000000000000000000000 P d418769ff6aa466ea11935ea8f4077d0 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d418769ff6aa466ea11935ea8f4077d0" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 42791 } }
I20260807 08:51:07.010567 29647 leader_election.cc:302] T 00000000000000000000000000000000 P d418769ff6aa466ea11935ea8f4077d0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d418769ff6aa466ea11935ea8f4077d0; no voters:
I20260807 08:51:07.010720 29647 leader_election.cc:288] T 00000000000000000000000000000000 P d418769ff6aa466ea11935ea8f4077d0 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260807 08:51:07.010771 29655 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d418769ff6aa466ea11935ea8f4077d0 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 08:51:07.010898 29655 raft_consensus.cc:686] T 00000000000000000000000000000000 P d418769ff6aa466ea11935ea8f4077d0 [term 3 LEADER]: Becoming Leader. State: Replica: d418769ff6aa466ea11935ea8f4077d0, State: Running, Role: LEADER
I20260807 08:51:07.010974 29647 sys_catalog.cc:564] T 00000000000000000000000000000000 P d418769ff6aa466ea11935ea8f4077d0 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:51:07.010983 29655 consensus_queue.cc:237] T 00000000000000000000000000000000 P d418769ff6aa466ea11935ea8f4077d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d418769ff6aa466ea11935ea8f4077d0" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 42791 } }
I20260807 08:51:07.011436 29658 sys_catalog.cc:455] T 00000000000000000000000000000000 P d418769ff6aa466ea11935ea8f4077d0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "d418769ff6aa466ea11935ea8f4077d0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d418769ff6aa466ea11935ea8f4077d0" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 42791 } } }
I20260807 08:51:07.011454 29656 sys_catalog.cc:455] T 00000000000000000000000000000000 P d418769ff6aa466ea11935ea8f4077d0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d418769ff6aa466ea11935ea8f4077d0. Latest consensus state: current_term: 3 leader_uuid: "d418769ff6aa466ea11935ea8f4077d0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d418769ff6aa466ea11935ea8f4077d0" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 42791 } } }
I20260807 08:51:07.011587 29658 sys_catalog.cc:458] T 00000000000000000000000000000000 P d418769ff6aa466ea11935ea8f4077d0 [sys.catalog]: This master's current role is: LEADER
I20260807 08:51:07.011608 29656 sys_catalog.cc:458] T 00000000000000000000000000000000 P d418769ff6aa466ea11935ea8f4077d0 [sys.catalog]: This master's current role is: LEADER
I20260807 08:51:07.012614 29683 catalog_manager.cc:1259] Loaded cluster ID: 4ab036fec3ac4624825127d7f6e296ab
I20260807 08:51:07.012655 29683 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d418769ff6aa466ea11935ea8f4077d0: loading cluster ID for follower catalog manager: success
I20260807 08:51:07.013446 29683 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d418769ff6aa466ea11935ea8f4077d0: acquiring CA information for follower catalog manager: success
F20260807 08:51:07.013754 29683 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f13df7ac9ad google::LogMessage::Fail() at ??:0
@ 0x7f13df7af88a google::LogMessage::SendToLog() at ??:0
@ 0x7f13df7ac4de google::LogMessage::Flush() at ??:0
@ 0x7f13df7ada19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f13e1f19104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f13e77056b9 kudu::master::CatalogManager::LoadTspkEntries() at ??:0
@ 0x7f13e7705989 kudu::master::CatalogManager::PrepareFollowerTokenVerifier() at ??:0
@ 0x7f13e7705e4e kudu::master::CatalogManager::PrepareFollower() at ??:0
@ 0x7f13e77088c2 kudu::master::CatalogManagerBgTasks::Run() at ??:0
@ 0x7f13e166f954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f13e24d36db start_thread at ??:0
@ 0x7f13de2aa61f clone at ??:0
W20260807 08:51:07.091403 29651 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:51:07.091601 29651 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:51:07.091634 29651 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:51:07.092724 29651 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:51:07.092763 29651 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:51:07.092796 29651 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.159.129
I20260807 08:51:07.094225 29651 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:33467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.129.keytab
--principal=kudu/127.4.159.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.159.129:42675
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.159.129
--webserver_port=35191
--tserver_master_addrs=127.4.159.190:42791
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29651
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.159.129
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:51:07.094457 29651 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:51:07.094646 29651 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:51:07.099236 29709 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:51:07.099347 29713 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:51:07.099228 29708 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:51:07.099478 29651 server_base.cc:1034] running on GCE node
I20260807 08:51:07.099650 29651 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:51:07.099885 29651 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:51:07.101027 29651 hybrid_clock.cc:648] HybridClock initialized: now 1786092667101020 us; error 34 us; skew 500 ppm
Aug 07 08:51:07 dist-test-slave-6zbq krb5kdc[27941](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092667, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:51:07.103274 29651 init.cc:377] Logged in from keytab as kudu/127.4.159.129@KRBTEST.COM (short username kudu)
I20260807 08:51:07.103669 29651 webserver.cc:460] Webserver started at http://127.4.159.129:35191/ using document root <none> and password file <none>
I20260807 08:51:07.103880 29651 fs_manager.cc:359] Metadata directory not provided
I20260807 08:51:07.103927 29651 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:51:07.105170 29651 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:51:07.105866 29726 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:51:07.106029 29651 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 08:51:07.106120 29651 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-0/wal
uuid: "0a8f9e196ee740fe9fdf8e9b935607d2"
format_stamp: "Formatted at 2026-08-07 08:50:25 on dist-test-slave-6zbq"
server_key: "682ec57daf6455a5f60b2065bed6d968"
server_key_iv: "4d2c0c2ea6783ca8e60a54dc0d34f9a9"
server_key_version: "encryptionkey@0"
I20260807 08:51:07.106567 29651 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:51:07.145448 29651 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:51:07.147428 29651 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:51:07.147557 29651 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:51:07.147758 29651 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:51:07.148181 29740 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260807 08:51:07.150000 29651 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 08:51:07.150064 29651 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260807 08:51:07.150102 29651 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 08:51:07.150790 29651 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 08:51:07.150825 29651 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260807 08:51:07.150909 29740 tablet_bootstrap.cc:492] T bac38484fd6c4ddfb722397f660a7a5f P 0a8f9e196ee740fe9fdf8e9b935607d2: Bootstrap starting.
I20260807 08:51:07.152714 29740 log.cc:824] T bac38484fd6c4ddfb722397f660a7a5f P 0a8f9e196ee740fe9fdf8e9b935607d2: Log is configured to *not* fsync() on all Append() calls
I20260807 08:51:07.155344 29740 tablet_bootstrap.cc:492] T bac38484fd6c4ddfb722397f660a7a5f P 0a8f9e196ee740fe9fdf8e9b935607d2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 08:51:07.155608 29740 tablet_bootstrap.cc:492] T bac38484fd6c4ddfb722397f660a7a5f P 0a8f9e196ee740fe9fdf8e9b935607d2: Bootstrap complete.
I20260807 08:51:07.155751 29740 ts_tablet_manager.cc:1397] T bac38484fd6c4ddfb722397f660a7a5f P 0a8f9e196ee740fe9fdf8e9b935607d2: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260807 08:51:07.157071 29651 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.129:42675
I20260807 08:51:07.157085 29904 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.129:42675 every 8 connection(s)
I20260807 08:51:07.157413 29651 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-0/data/info.pb
I20260807 08:51:07.157567 29740 raft_consensus.cc:348] T bac38484fd6c4ddfb722397f660a7a5f P 0a8f9e196ee740fe9fdf8e9b935607d2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf081cf53efe44d5b6770c9d2fbf017b" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 45125 } } peers { permanent_uuid: "bfbbc2d0d72b4f628ef3f2522c9c2106" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 45693 } } peers { permanent_uuid: "0a8f9e196ee740fe9fdf8e9b935607d2" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 42675 } }
I20260807 08:51:07.157979 29740 raft_consensus.cc:729] T bac38484fd6c4ddfb722397f660a7a5f P 0a8f9e196ee740fe9fdf8e9b935607d2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a8f9e196ee740fe9fdf8e9b935607d2, State: Initialized, Role: FOLLOWER
I20260807 08:51:07.158120 29740 consensus_queue.cc:260] T bac38484fd6c4ddfb722397f660a7a5f P 0a8f9e196ee740fe9fdf8e9b935607d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf081cf53efe44d5b6770c9d2fbf017b" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 45125 } } peers { permanent_uuid: "bfbbc2d0d72b4f628ef3f2522c9c2106" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 45693 } } peers { permanent_uuid: "0a8f9e196ee740fe9fdf8e9b935607d2" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 42675 } }
I20260807 08:51:07.158394 29740 ts_tablet_manager.cc:1428] T bac38484fd6c4ddfb722397f660a7a5f P 0a8f9e196ee740fe9fdf8e9b935607d2: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260807 08:51:07.164484 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 29651
I20260807 08:51:07.164728 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.159.130:45125
--local_ip_for_outbound_sockets=127.4.159.130
--tserver_master_addrs=127.4.159.190:42791
--webserver_port=45979
--webserver_interface=127.4.159.130
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.130.keytab
--principal=kudu/127.4.159.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:33467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:51:07.205323 29908 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:51:07.158009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:42791 (local address 127.4.159.129:57053)
0807 08:51:07.158178 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:07.158211 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0807 08:51:07.158388 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:51:07.205129 (+ 46741us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:42791: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":78,"threads_started":1}
W20260807 08:51:07.205567 29905 heartbeater.cc:644] Failed to heartbeat to 127.4.159.190:42791 (0 consecutive failures): Network error: Failed to ping master at 127.4.159.190:42791: Client connection negotiation failed: client connection to 127.4.159.190:42791: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 08:51:07.205889 29908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:07.205766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:42791 (local address 127.4.159.129:37471)
0807 08:51:07.205827 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:07.205866 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:42791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260807 08:51:07.206174 29908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:07.206093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:42791 (local address 127.4.159.129:50903)
0807 08:51:07.206131 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:07.206155 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:42791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260807 08:51:07.206305 29905 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 08:51:07.257189 29910 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:51:07.257366 29910 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:51:07.257387 29910 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:51:07.258505 29910 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:51:07.258541 29910 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:51:07.258565 29910 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.159.130
I20260807 08:51:07.259948 29910 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:33467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.130.keytab
--principal=kudu/127.4.159.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.159.130:45125
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.159.130
--webserver_port=45979
--tserver_master_addrs=127.4.159.190:42791
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29910
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.159.130
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:51:07.260165 29910 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:51:07.260353 29910 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:51:07.264835 29917 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:51:07.264935 29910 server_base.cc:1034] running on GCE node
W20260807 08:51:07.264837 29920 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:51:07.264889 29918 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:51:07.265197 29910 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:51:07.265417 29910 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:51:07.266541 29910 hybrid_clock.cc:648] HybridClock initialized: now 1786092667266530 us; error 27 us; skew 500 ppm
Aug 07 08:51:07 dist-test-slave-6zbq krb5kdc[27941](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092667, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:51:07.268827 29910 init.cc:377] Logged in from keytab as kudu/127.4.159.130@KRBTEST.COM (short username kudu)
I20260807 08:51:07.269214 29910 webserver.cc:460] Webserver started at http://127.4.159.130:45979/ using document root <none> and password file <none>
I20260807 08:51:07.269373 29910 fs_manager.cc:359] Metadata directory not provided
I20260807 08:51:07.269410 29910 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:51:07.270572 29910 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:51:07.271104 29934 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:51:07.271243 29910 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 08:51:07.271317 29910 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-1/wal
uuid: "bf081cf53efe44d5b6770c9d2fbf017b"
format_stamp: "Formatted at 2026-08-07 08:50:25 on dist-test-slave-6zbq"
server_key: "0bb9e0da7bc822a4aed2edf933475ba7"
server_key_iv: "24a289caf7b807a8e7434edbe7a6aaf6"
server_key_version: "encryptionkey@0"
I20260807 08:51:07.271600 29910 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:51:07.308555 29910 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:51:07.310168 29910 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:51:07.310285 29910 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:51:07.310521 29910 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:51:07.310933 29943 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260807 08:51:07.312304 29910 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 08:51:07.312343 29910 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260807 08:51:07.312366 29910 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 08:51:07.312990 29910 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 08:51:07.313022 29910 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260807 08:51:07.313078 29943 tablet_bootstrap.cc:492] T bac38484fd6c4ddfb722397f660a7a5f P bf081cf53efe44d5b6770c9d2fbf017b: Bootstrap starting.
I20260807 08:51:07.318821 29910 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.130:45125
I20260807 08:51:07.318905 30116 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.130:45125 every 8 connection(s)
I20260807 08:51:07.319168 29910 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-1/data/info.pb
I20260807 08:51:07.320183 30120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:07.319715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:42791 (local address 127.4.159.130:34603)
0807 08:51:07.319890 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:07.319984 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:42791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":79,"threads_started":1}
W20260807 08:51:07.320384 30117 heartbeater.cc:644] Failed to heartbeat to 127.4.159.190:42791 (0 consecutive failures): Network error: Failed to ping master at 127.4.159.190:42791: Client connection negotiation failed: client connection to 127.4.159.190:42791: connect: Connection refused (error 111)
I20260807 08:51:07.320647 30120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:07.320553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:42791 (local address 127.4.159.130:47261)
0807 08:51:07.320596 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:07.320625 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:42791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260807 08:51:07.320920 30120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:07.320831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:42791 (local address 127.4.159.130:43727)
0807 08:51:07.320874 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:07.320900 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:42791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260807 08:51:07.321024 30117 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 08:51:07.322696 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 29910
I20260807 08:51:07.322930 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.159.131:45693
--local_ip_for_outbound_sockets=127.4.159.131
--tserver_master_addrs=127.4.159.190:42791
--webserver_port=46805
--webserver_interface=127.4.159.131
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.131.keytab
--principal=kudu/127.4.159.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:33467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 08:51:07.323168 29943 log.cc:824] T bac38484fd6c4ddfb722397f660a7a5f P bf081cf53efe44d5b6770c9d2fbf017b: Log is configured to *not* fsync() on all Append() calls
I20260807 08:51:07.325369 29943 tablet_bootstrap.cc:492] T bac38484fd6c4ddfb722397f660a7a5f P bf081cf53efe44d5b6770c9d2fbf017b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 08:51:07.325624 29943 tablet_bootstrap.cc:492] T bac38484fd6c4ddfb722397f660a7a5f P bf081cf53efe44d5b6770c9d2fbf017b: Bootstrap complete.
I20260807 08:51:07.326398 29943 ts_tablet_manager.cc:1397] T bac38484fd6c4ddfb722397f660a7a5f P bf081cf53efe44d5b6770c9d2fbf017b: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.008s
I20260807 08:51:07.327517 29943 raft_consensus.cc:348] T bac38484fd6c4ddfb722397f660a7a5f P bf081cf53efe44d5b6770c9d2fbf017b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf081cf53efe44d5b6770c9d2fbf017b" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 45125 } } peers { permanent_uuid: "bfbbc2d0d72b4f628ef3f2522c9c2106" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 45693 } } peers { permanent_uuid: "0a8f9e196ee740fe9fdf8e9b935607d2" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 42675 } }
I20260807 08:51:07.327756 29943 raft_consensus.cc:729] T bac38484fd6c4ddfb722397f660a7a5f P bf081cf53efe44d5b6770c9d2fbf017b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf081cf53efe44d5b6770c9d2fbf017b, State: Initialized, Role: FOLLOWER
I20260807 08:51:07.327864 29943 consensus_queue.cc:260] T bac38484fd6c4ddfb722397f660a7a5f P bf081cf53efe44d5b6770c9d2fbf017b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf081cf53efe44d5b6770c9d2fbf017b" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 45125 } } peers { permanent_uuid: "bfbbc2d0d72b4f628ef3f2522c9c2106" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 45693 } } peers { permanent_uuid: "0a8f9e196ee740fe9fdf8e9b935607d2" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 42675 } }
I20260807 08:51:07.328086 29943 ts_tablet_manager.cc:1428] T bac38484fd6c4ddfb722397f660a7a5f P bf081cf53efe44d5b6770c9d2fbf017b: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260807 08:51:07.328292 30120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:07.328208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:42791 (local address 127.4.159.130:39589)
0807 08:51:07.328246 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:07.328270 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:42791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260807 08:51:07.328544 30120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:07.328465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:42791 (local address 127.4.159.130:54391)
0807 08:51:07.328502 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:07.328524 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:42791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260807 08:51:07.403338 30122 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:51:07.403518 30122 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:51:07.403549 30122 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:51:07.404630 30122 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:51:07.404668 30122 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:51:07.404702 30122 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.159.131
I20260807 08:51:07.406100 30122 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:33467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.131.keytab
--principal=kudu/127.4.159.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.159.131:45693
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.159.131
--webserver_port=46805
--tserver_master_addrs=127.4.159.190:42791
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30122
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.159.131
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:51:07.406318 30122 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:51:07.406533 30122 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:51:07.411033 30133 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:51:07.411118 30122 server_base.cc:1034] running on GCE node
W20260807 08:51:07.411065 30130 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:51:07.411036 30131 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:51:07.411474 30122 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:51:07.411695 30122 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:51:07.412837 30122 hybrid_clock.cc:648] HybridClock initialized: now 1786092667412823 us; error 31 us; skew 500 ppm
Aug 07 08:51:07 dist-test-slave-6zbq krb5kdc[27941](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092667, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:51:07.415148 30122 init.cc:377] Logged in from keytab as kudu/127.4.159.131@KRBTEST.COM (short username kudu)
I20260807 08:51:07.415555 30122 webserver.cc:460] Webserver started at http://127.4.159.131:46805/ using document root <none> and password file <none>
I20260807 08:51:07.415743 30122 fs_manager.cc:359] Metadata directory not provided
I20260807 08:51:07.415788 30122 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:51:07.416975 30122 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:51:07.417474 30145 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:51:07.417603 30122 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 08:51:07.417682 30122 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-2/wal
uuid: "bfbbc2d0d72b4f628ef3f2522c9c2106"
format_stamp: "Formatted at 2026-08-07 08:50:25 on dist-test-slave-6zbq"
server_key: "2444b091a0eb01fa0372129703c1dae2"
server_key_iv: "826f8fdd564a007314aa095676b3776d"
server_key_version: "encryptionkey@0"
I20260807 08:51:07.417963 30122 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:51:07.433569 30122 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:51:07.435184 30122 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:51:07.435314 30122 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:51:07.435511 30122 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:51:07.435776 30122 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:51:07.435814 30122 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:51:07.435840 30122 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:51:07.435856 30122 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:51:07.441095 30122 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.131:45693
I20260807 08:51:07.441148 30334 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.131:45693 every 8 connection(s)
I20260807 08:51:07.441435 30122 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786092453486433-4734-0/minicluster-data/ts-2/data/info.pb
I20260807 08:51:07.442291 30337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:07.441971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:42791 (local address 127.4.159.131:35393)
0807 08:51:07.442104 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:07.442166 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:42791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":68,"threads_started":1}
W20260807 08:51:07.442625 30335 heartbeater.cc:644] Failed to heartbeat to 127.4.159.190:42791 (0 consecutive failures): Network error: Failed to ping master at 127.4.159.190:42791: Client connection negotiation failed: client connection to 127.4.159.190:42791: connect: Connection refused (error 111)
I20260807 08:51:07.442847 30337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:07.442770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:42791 (local address 127.4.159.131:40711)
0807 08:51:07.442808 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:07.442831 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:42791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260807 08:51:07.443099 30337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:07.443028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:42791 (local address 127.4.159.131:56565)
0807 08:51:07.443064 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:07.443084 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:42791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260807 08:51:07.443194 30335 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 08:51:07.449170 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 30122
I20260807 08:51:08.206892 30339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:08.206661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:42791 (local address 127.4.159.129:45665)
0807 08:51:08.206796 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:08.206830 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:42791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":76,"threads_started":1}
I20260807 08:51:08.329046 30340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:08.328855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:42791 (local address 127.4.159.130:48319)
0807 08:51:08.328967 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:08.329002 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:42791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":54,"threads_started":1}
I20260807 08:51:08.443681 30341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:08.443461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:42791 (local address 127.4.159.131:42177)
0807 08:51:08.443589 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:08.443621 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:42791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":57,"threads_started":1}
I20260807 08:51:08.682904 30342 raft_consensus.cc:482] T bac38484fd6c4ddfb722397f660a7a5f P bf081cf53efe44d5b6770c9d2fbf017b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:51:08.683018 30342 raft_consensus.cc:504] T bac38484fd6c4ddfb722397f660a7a5f P bf081cf53efe44d5b6770c9d2fbf017b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf081cf53efe44d5b6770c9d2fbf017b" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 45125 } } peers { permanent_uuid: "bfbbc2d0d72b4f628ef3f2522c9c2106" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 45693 } } peers { permanent_uuid: "0a8f9e196ee740fe9fdf8e9b935607d2" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 42675 } }
I20260807 08:51:08.683307 30342 leader_election.cc:288] T bac38484fd6c4ddfb722397f660a7a5f P bf081cf53efe44d5b6770c9d2fbf017b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bfbbc2d0d72b4f628ef3f2522c9c2106 (127.4.159.131:45693), 0a8f9e196ee740fe9fdf8e9b935607d2 (127.4.159.129:42675)
Aug 07 08:51:08 dist-test-slave-6zbq krb5kdc[27941](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092667, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.130@KRBTEST.COM for kudu/127.4.159.131@KRBTEST.COM
Aug 07 08:51:08 dist-test-slave-6zbq krb5kdc[27941](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092667, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.130@KRBTEST.COM for kudu/127.4.159.129@KRBTEST.COM
I20260807 08:51:08.692864 30346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:08.683597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.130:50637 (local address 127.4.159.129:42675)
0807 08:51:08.683734 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:08.683740 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:08.683753 (+ 13us) server_negotiation.cc:407] Connection header received
0807 08:51:08.683876 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:08.683878 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:08.683900 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:08.683997 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:51:08.684680 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:08.685538 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:08.686558 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:08.686954 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:08.687053 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:08.689551 (+ 2498us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:51:08.689554 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:51:08.689554 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:51:08.689564 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:51:08.691498 (+ 1934us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:51:08.691830 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:51:08.691833 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:51:08.691833 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:51:08.691937 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:51:08.692107 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:51:08.692108 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:51:08.692108 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:51:08.692378 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:51:08.692447 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:08.692637 (+ 190us) server_negotiation.cc:299] Negotiation successful
0807 08:51:08.692758 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":51,"threads_started":1}
I20260807 08:51:08.692871 30345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:08.683557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.130:40627 (local address 127.4.159.131:45693)
0807 08:51:08.683737 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:08.683747 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:08.683784 (+ 37us) server_negotiation.cc:407] Connection header received
0807 08:51:08.684298 (+ 514us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:08.684301 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:08.684340 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:08.684439 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:51:08.684736 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:08.685560 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:08.686565 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:08.686956 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:08.687053 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:08.689227 (+ 2174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:51:08.689230 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:51:08.689230 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:51:08.689241 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:51:08.691040 (+ 1799us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:51:08.691418 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:51:08.691420 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:51:08.691421 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:51:08.691528 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:51:08.691954 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:51:08.691956 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:51:08.691956 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:51:08.692272 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:51:08.692374 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:08.692638 (+ 264us) server_negotiation.cc:299] Negotiation successful
0807 08:51:08.692760 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":81,"threads_started":1}
I20260807 08:51:08.692902 30340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:08.683454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.131:45693 (local address 127.4.159.130:40627)
0807 08:51:08.683515 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:08.683543 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0807 08:51:08.683735 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:51:08.684456 (+ 721us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:51:08.684457 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:51:08.684598 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:51:08.684637 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:08.684638 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:08.685648 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:08.685649 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:08.686479 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:08.686482 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:08.687034 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:08.687035 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:08.687150 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:08.687701 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:51:08.687706 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:51:08.689124 (+ 1418us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:51:08.691113 (+ 1989us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:51:08.691119 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:51:08.691121 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:51:08.691341 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:51:08.691597 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:51:08.691597 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:51:08.691598 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:51:08.691894 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:51:08.692337 (+ 443us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:51:08.692342 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:51:08.692479 (+ 137us) client_negotiation.cc:769] Sending connection context
0807 08:51:08.692569 (+ 90us) client_negotiation.cc:240] Negotiation successful
0807 08:51:08.692759 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45,"mutex_wait_us":46}
I20260807 08:51:08.693219 30343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:08.683460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.129:42675 (local address 127.4.159.130:50637)
0807 08:51:08.683598 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:08.683607 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 08:51:08.683746 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:51:08.684028 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:51:08.684031 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:51:08.684309 (+ 278us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:51:08.684546 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:08.684555 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:08.685639 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:08.685640 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:08.686479 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:08.686481 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:08.687033 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:08.687034 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:08.687151 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:08.687649 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:51:08.687665 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:51:08.689459 (+ 1794us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:51:08.691565 (+ 2106us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:51:08.691566 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:51:08.691566 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:51:08.691751 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:51:08.692003 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:51:08.692003 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:51:08.692003 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:51:08.692054 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:51:08.692434 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:51:08.692435 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:51:08.692485 (+ 50us) client_negotiation.cc:769] Sending connection context
0807 08:51:08.692568 (+ 83us) client_negotiation.cc:240] Negotiation successful
0807 08:51:08.692835 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"mutex_wait_us":150,"thread_start_us":43,"threads_started":1}
I20260807 08:51:08.693598 29830 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bac38484fd6c4ddfb722397f660a7a5f" candidate_uuid: "bf081cf53efe44d5b6770c9d2fbf017b" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "0a8f9e196ee740fe9fdf8e9b935607d2" is_pre_election: true
W20260807 08:51:08.693676 29938 leader_election.cc:341] T bac38484fd6c4ddfb722397f660a7a5f P bf081cf53efe44d5b6770c9d2fbf017b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bfbbc2d0d72b4f628ef3f2522c9c2106 (127.4.159.131:45693): Not found: Tablet not found: bac38484fd6c4ddfb722397f660a7a5f
I20260807 08:51:08.693748 29830 raft_consensus.cc:2454] T bac38484fd6c4ddfb722397f660a7a5f P 0a8f9e196ee740fe9fdf8e9b935607d2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf081cf53efe44d5b6770c9d2fbf017b in term 1.
I20260807 08:51:08.693974 29940 leader_election.cc:302] T bac38484fd6c4ddfb722397f660a7a5f P bf081cf53efe44d5b6770c9d2fbf017b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a8f9e196ee740fe9fdf8e9b935607d2, bf081cf53efe44d5b6770c9d2fbf017b; no voters: bfbbc2d0d72b4f628ef3f2522c9c2106
I20260807 08:51:08.694088 30342 raft_consensus.cc:2781] T bac38484fd6c4ddfb722397f660a7a5f P bf081cf53efe44d5b6770c9d2fbf017b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 08:51:08.694149 30342 raft_consensus.cc:482] T bac38484fd6c4ddfb722397f660a7a5f P bf081cf53efe44d5b6770c9d2fbf017b [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:51:08.694172 30342 raft_consensus.cc:3037] T bac38484fd6c4ddfb722397f660a7a5f P bf081cf53efe44d5b6770c9d2fbf017b [term 1 FOLLOWER]: Advancing to term 2
I20260807 08:51:08.695259 30342 raft_consensus.cc:504] T bac38484fd6c4ddfb722397f660a7a5f P bf081cf53efe44d5b6770c9d2fbf017b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf081cf53efe44d5b6770c9d2fbf017b" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 45125 } } peers { permanent_uuid: "bfbbc2d0d72b4f628ef3f2522c9c2106" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 45693 } } peers { permanent_uuid: "0a8f9e196ee740fe9fdf8e9b935607d2" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 42675 } }
I20260807 08:51:08.695418 30342 leader_election.cc:288] T bac38484fd6c4ddfb722397f660a7a5f P bf081cf53efe44d5b6770c9d2fbf017b [CANDIDATE]: Term 2 election: Requested vote from peers bfbbc2d0d72b4f628ef3f2522c9c2106 (127.4.159.131:45693), 0a8f9e196ee740fe9fdf8e9b935607d2 (127.4.159.129:42675)
I20260807 08:51:08.695590 29830 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bac38484fd6c4ddfb722397f660a7a5f" candidate_uuid: "bf081cf53efe44d5b6770c9d2fbf017b" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "0a8f9e196ee740fe9fdf8e9b935607d2"
I20260807 08:51:08.695672 29830 raft_consensus.cc:3037] T bac38484fd6c4ddfb722397f660a7a5f P 0a8f9e196ee740fe9fdf8e9b935607d2 [term 1 FOLLOWER]: Advancing to term 2
W20260807 08:51:08.695691 29938 leader_election.cc:341] T bac38484fd6c4ddfb722397f660a7a5f P bf081cf53efe44d5b6770c9d2fbf017b [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer bfbbc2d0d72b4f628ef3f2522c9c2106 (127.4.159.131:45693): Not found: Tablet not found: bac38484fd6c4ddfb722397f660a7a5f
I20260807 08:51:08.696708 29830 raft_consensus.cc:2454] T bac38484fd6c4ddfb722397f660a7a5f P 0a8f9e196ee740fe9fdf8e9b935607d2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf081cf53efe44d5b6770c9d2fbf017b in term 2.
I20260807 08:51:08.696856 29940 leader_election.cc:302] T bac38484fd6c4ddfb722397f660a7a5f P bf081cf53efe44d5b6770c9d2fbf017b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a8f9e196ee740fe9fdf8e9b935607d2, bf081cf53efe44d5b6770c9d2fbf017b; no voters: bfbbc2d0d72b4f628ef3f2522c9c2106
I20260807 08:51:08.696938 30342 raft_consensus.cc:2781] T bac38484fd6c4ddfb722397f660a7a5f P bf081cf53efe44d5b6770c9d2fbf017b [term 2 FOLLOWER]: Leader election won for term 2
I20260807 08:51:08.697041 30342 raft_consensus.cc:686] T bac38484fd6c4ddfb722397f660a7a5f P bf081cf53efe44d5b6770c9d2fbf017b [term 2 LEADER]: Becoming Leader. State: Replica: bf081cf53efe44d5b6770c9d2fbf017b, State: Running, Role: LEADER
I20260807 08:51:08.697116 30342 consensus_queue.cc:237] T bac38484fd6c4ddfb722397f660a7a5f P bf081cf53efe44d5b6770c9d2fbf017b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf081cf53efe44d5b6770c9d2fbf017b" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 45125 } } peers { permanent_uuid: "bfbbc2d0d72b4f628ef3f2522c9c2106" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 45693 } } peers { permanent_uuid: "0a8f9e196ee740fe9fdf8e9b935607d2" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 42675 } }
I20260807 08:51:08.697362 30343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:08.697210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:42791 (local address 127.4.159.130:34239)
0807 08:51:08.697307 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:08.697339 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:42791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
W20260807 08:51:09.078888 29938 consensus_peers.cc:487] T bac38484fd6c4ddfb722397f660a7a5f P bf081cf53efe44d5b6770c9d2fbf017b -> Peer bfbbc2d0d72b4f628ef3f2522c9c2106 (127.4.159.131:45693): Couldn't send request to peer bfbbc2d0d72b4f628ef3f2522c9c2106. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bac38484fd6c4ddfb722397f660a7a5f. This is attempt 1: this message will repeat every 5th retry.
I20260807 08:51:09.192248 29830 raft_consensus.cc:1261] T bac38484fd6c4ddfb722397f660a7a5f P 0a8f9e196ee740fe9fdf8e9b935607d2 [term 2 FOLLOWER]: Refusing update from remote peer bf081cf53efe44d5b6770c9d2fbf017b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260807 08:51:09.192590 30342 consensus_queue.cc:1035] T bac38484fd6c4ddfb722397f660a7a5f P bf081cf53efe44d5b6770c9d2fbf017b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a8f9e196ee740fe9fdf8e9b935607d2" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 08:51:09.193445 30353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:09.193282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:42791 (local address 127.4.159.129:55039)
0807 08:51:09.193375 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:09.193409 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:42791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83,"thread_start_us":40,"threads_started":1}
I20260807 08:51:09.194141 30343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:09.194028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:42791 (local address 127.4.159.130:47501)
0807 08:51:09.194070 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:09.194095 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:42791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260807 08:51:09.194212 30353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:09.194127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:42791 (local address 127.4.159.129:59203)
0807 08:51:09.194158 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:09.194183 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:42791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
I20260807 08:51:09.444244 30356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:09.444052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:42791 (local address 127.4.159.131:36183)
0807 08:51:09.444175 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:09.444207 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:42791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":93,"threads_started":1}
I20260807 08:51:09.536790 30357 ts_tablet_manager.cc:927] T bac38484fd6c4ddfb722397f660a7a5f P bfbbc2d0d72b4f628ef3f2522c9c2106: Initiating tablet copy from peer bf081cf53efe44d5b6770c9d2fbf017b (127.4.159.130:45125)
I20260807 08:51:09.537009 30357 tablet_copy_client.cc:316] T bac38484fd6c4ddfb722397f660a7a5f P bfbbc2d0d72b4f628ef3f2522c9c2106: tablet copy: Beginning tablet copy session from remote peer at address 127.4.159.130:45125
Aug 07 08:51:09 dist-test-slave-6zbq krb5kdc[27941](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092667, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.131@KRBTEST.COM for kudu/127.4.159.130@KRBTEST.COM
I20260807 08:51:09.544425 30359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:09.537349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.131:56163 (local address 127.4.159.130:45125)
0807 08:51:09.537538 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:09.537542 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:09.537550 (+ 8us) server_negotiation.cc:407] Connection header received
0807 08:51:09.537573 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:09.537575 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:09.537588 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:09.537651 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:51:09.538110 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:09.538841 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:09.539577 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:09.539875 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:09.539935 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:09.541877 (+ 1942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:51:09.541886 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:51:09.541886 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:51:09.541892 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:51:09.543279 (+ 1387us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:51:09.543634 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:51:09.543638 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:51:09.543638 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:51:09.543695 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:51:09.543861 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:51:09.543862 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:51:09.543863 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:51:09.544005 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:51:09.544075 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:09.544246 (+ 171us) server_negotiation.cc:299] Negotiation successful
0807 08:51:09.544325 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":101,"threads_started":1}
I20260807 08:51:09.544441 30356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:09.537247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.130:45125 (local address 127.4.159.131:56163)
0807 08:51:09.537298 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:09.537318 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 08:51:09.537388 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:51:09.537684 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:51:09.537691 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:51:09.537970 (+ 279us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:51:09.538032 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:09.538034 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:09.538929 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:09.538929 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:09.539498 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:09.539500 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:09.539943 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:09.539944 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:09.540017 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:09.540484 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:51:09.540488 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:51:09.541793 (+ 1305us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:51:09.543359 (+ 1566us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:51:09.543366 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:51:09.543368 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:51:09.543558 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:51:09.543751 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:51:09.543752 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:51:09.543752 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:51:09.543811 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:51:09.544066 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:51:09.544073 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:51:09.544132 (+ 59us) client_negotiation.cc:769] Sending connection context
0807 08:51:09.544206 (+ 74us) client_negotiation.cc:240] Negotiation successful
0807 08:51:09.544323 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260807 08:51:09.544838 30073 tablet_copy_service.cc:140] P bf081cf53efe44d5b6770c9d2fbf017b: Received BeginTabletCopySession request for tablet bac38484fd6c4ddfb722397f660a7a5f from peer bfbbc2d0d72b4f628ef3f2522c9c2106 ({username='kudu', principal='kudu/127.4.159.131@KRBTEST.COM'} at 127.4.159.131:56163)
I20260807 08:51:09.544899 30073 tablet_copy_service.cc:161] P bf081cf53efe44d5b6770c9d2fbf017b: Beginning new tablet copy session on tablet bac38484fd6c4ddfb722397f660a7a5f from peer bfbbc2d0d72b4f628ef3f2522c9c2106 at {username='kudu', principal='kudu/127.4.159.131@KRBTEST.COM'} at 127.4.159.131:56163: session id = bfbbc2d0d72b4f628ef3f2522c9c2106-bac38484fd6c4ddfb722397f660a7a5f
I20260807 08:51:09.545344 30073 tablet_copy_source_session.cc:205] T bac38484fd6c4ddfb722397f660a7a5f P bf081cf53efe44d5b6770c9d2fbf017b: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 08:51:09.546214 30357 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bac38484fd6c4ddfb722397f660a7a5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:51:09.548840 30357 tablet_copy_client.cc:791] T bac38484fd6c4ddfb722397f660a7a5f P bfbbc2d0d72b4f628ef3f2522c9c2106: tablet copy: Starting download of 0 data blocks...
I20260807 08:51:09.548969 30357 tablet_copy_client.cc:655] T bac38484fd6c4ddfb722397f660a7a5f P bfbbc2d0d72b4f628ef3f2522c9c2106: tablet copy: Starting download of 1 WAL segments...
I20260807 08:51:09.549749 30357 tablet_copy_client.cc:531] T bac38484fd6c4ddfb722397f660a7a5f P bfbbc2d0d72b4f628ef3f2522c9c2106: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 08:51:09.550427 30357 tablet_bootstrap.cc:492] T bac38484fd6c4ddfb722397f660a7a5f P bfbbc2d0d72b4f628ef3f2522c9c2106: Bootstrap starting.
I20260807 08:51:09.551651 30357 log.cc:824] T bac38484fd6c4ddfb722397f660a7a5f P bfbbc2d0d72b4f628ef3f2522c9c2106: Log is configured to *not* fsync() on all Append() calls
I20260807 08:51:09.554016 30357 tablet_bootstrap.cc:492] T bac38484fd6c4ddfb722397f660a7a5f P bfbbc2d0d72b4f628ef3f2522c9c2106: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 08:51:09.554214 30357 tablet_bootstrap.cc:492] T bac38484fd6c4ddfb722397f660a7a5f P bfbbc2d0d72b4f628ef3f2522c9c2106: Bootstrap complete.
I20260807 08:51:09.554323 30357 ts_tablet_manager.cc:1397] T bac38484fd6c4ddfb722397f660a7a5f P bfbbc2d0d72b4f628ef3f2522c9c2106: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260807 08:51:09.555228 30357 raft_consensus.cc:348] T bac38484fd6c4ddfb722397f660a7a5f P bfbbc2d0d72b4f628ef3f2522c9c2106 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf081cf53efe44d5b6770c9d2fbf017b" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 45125 } } peers { permanent_uuid: "bfbbc2d0d72b4f628ef3f2522c9c2106" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 45693 } } peers { permanent_uuid: "0a8f9e196ee740fe9fdf8e9b935607d2" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 42675 } }
I20260807 08:51:09.555361 30357 raft_consensus.cc:729] T bac38484fd6c4ddfb722397f660a7a5f P bfbbc2d0d72b4f628ef3f2522c9c2106 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfbbc2d0d72b4f628ef3f2522c9c2106, State: Initialized, Role: FOLLOWER
I20260807 08:51:09.555460 30357 consensus_queue.cc:260] T bac38484fd6c4ddfb722397f660a7a5f P bfbbc2d0d72b4f628ef3f2522c9c2106 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf081cf53efe44d5b6770c9d2fbf017b" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 45125 } } peers { permanent_uuid: "bfbbc2d0d72b4f628ef3f2522c9c2106" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 45693 } } peers { permanent_uuid: "0a8f9e196ee740fe9fdf8e9b935607d2" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 42675 } }
I20260807 08:51:09.555692 30357 ts_tablet_manager.cc:1428] T bac38484fd6c4ddfb722397f660a7a5f P bfbbc2d0d72b4f628ef3f2522c9c2106: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260807 08:51:09.555955 30356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:09.555856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:42791 (local address 127.4.159.131:38675)
0807 08:51:09.555899 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:09.555931 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:42791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260807 08:51:09.556020 30073 tablet_copy_service.cc:342] P bf081cf53efe44d5b6770c9d2fbf017b: Request end of tablet copy session bfbbc2d0d72b4f628ef3f2522c9c2106-bac38484fd6c4ddfb722397f660a7a5f received from {username='kudu', principal='kudu/127.4.159.131@KRBTEST.COM'} at 127.4.159.131:56163
I20260807 08:51:09.556107 30073 tablet_copy_service.cc:433] P bf081cf53efe44d5b6770c9d2fbf017b: ending tablet copy session bfbbc2d0d72b4f628ef3f2522c9c2106-bac38484fd6c4ddfb722397f660a7a5f on tablet bac38484fd6c4ddfb722397f660a7a5f with peer bfbbc2d0d72b4f628ef3f2522c9c2106
I20260807 08:51:09.556263 30356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:09.556183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:42791 (local address 127.4.159.131:51413)
0807 08:51:09.556221 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:09.556244 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:42791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260807 08:51:10.070842 30366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:10.070670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:42791 (local address 127.4.159.130:60675)
0807 08:51:10.070763 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:10.070796 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:42791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79,"thread_start_us":52,"threads_started":1}
I20260807 08:51:10.070827 30250 raft_consensus.cc:1203] T bac38484fd6c4ddfb722397f660a7a5f P bfbbc2d0d72b4f628ef3f2522c9c2106 [term 2 FOLLOWER]: Deduplicated request from leader. Original: 1.2->[2.3-2.3] Dedup: 2.3->[]
I20260807 08:51:10.071324 30369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:10.071179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:42791 (local address 127.4.159.131:49741)
0807 08:51:10.071264 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:10.071297 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:42791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70,"thread_start_us":42,"threads_started":1}
I20260807 08:51:10.194718 30370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:10.194520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:42791 (local address 127.4.159.129:47099)
0807 08:51:10.194636 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:10.194671 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:42791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":53,"threads_started":1}
I20260807 08:51:11.071420 30371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:11.071204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:42791 (local address 127.4.159.130:40145)
0807 08:51:11.071341 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:11.071375 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:42791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":62,"threads_started":1}
I20260807 08:51:11.071861 30372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:11.071679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:42791 (local address 127.4.159.131:43549)
0807 08:51:11.071795 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:11.071823 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:42791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":62,"threads_started":1}
I20260807 08:51:11.195247 30373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:11.195059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:42791 (local address 127.4.159.129:34207)
0807 08:51:11.195171 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:11.195204 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:42791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":62,"threads_started":1}
I20260807 08:51:12.071990 30374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:12.071796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:42791 (local address 127.4.159.130:51573)
0807 08:51:12.071912 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:12.071945 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:42791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":59,"threads_started":1}
I20260807 08:51:12.072422 30375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:12.072237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:42791 (local address 127.4.159.131:53725)
0807 08:51:12.072355 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:12.072383 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:42791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":64,"threads_started":1}
I20260807 08:51:12.195890 30376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:12.195698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:42791 (local address 127.4.159.129:34069)
0807 08:51:12.195815 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:12.195847 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:42791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":62,"threads_started":1}
I20260807 08:51:13.072656 30378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:13.072436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:42791 (local address 127.4.159.130:35455)
0807 08:51:13.072571 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:13.072608 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:42791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":75,"threads_started":1}
I20260807 08:51:13.073009 30379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:13.072808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:42791 (local address 127.4.159.131:60163)
0807 08:51:13.072929 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:13.072963 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:42791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":66,"threads_started":1}
I20260807 08:51:13.073238 30378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:13.073074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:42791 (local address 127.4.159.130:54011)
0807 08:51:13.073183 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:13.073210 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:42791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260807 08:51:13.196488 30381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:13.196271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:42791 (local address 127.4.159.129:42393)
0807 08:51:13.196407 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:13.196443 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:42791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":59,"threads_started":1}
I20260807 08:51:14.073619 30384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:14.073416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:42791 (local address 127.4.159.131:53293)
0807 08:51:14.073539 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:14.073573 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:42791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":66,"threads_started":1}
I20260807 08:51:14.073750 30385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:14.073567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:42791 (local address 127.4.159.130:57799)
0807 08:51:14.073665 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:14.073698 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:42791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82,"thread_start_us":44,"threads_started":1}
I20260807 08:51:14.197110 30387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:14.196902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:42791 (local address 127.4.159.129:51517)
0807 08:51:14.197030 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:14.197064 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:42791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":76,"threads_started":1}
I20260807 08:51:15.074316 30388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:15.074076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:42791 (local address 127.4.159.131:42503)
0807 08:51:15.074213 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:15.074260 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:42791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":65,"threads_started":1}
I20260807 08:51:15.074405 30389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:15.074118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:42791 (local address 127.4.159.130:35185)
0807 08:51:15.074295 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:15.074332 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:42791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":116,"threads_started":1}
I20260807 08:51:15.197746 30391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:15.197532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:42791 (local address 127.4.159.129:50525)
0807 08:51:15.197660 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:15.197696 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:42791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":62,"threads_started":1}
I20260807 08:51:16.074980 30393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:16.074759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:42791 (local address 127.4.159.131:56235)
0807 08:51:16.074897 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:16.074934 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:42791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":83,"threads_started":1}
I20260807 08:51:16.074990 30394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:16.074810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:42791 (local address 127.4.159.130:44647)
0807 08:51:16.074913 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:16.074946 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:42791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87,"thread_start_us":61,"threads_started":1}
I20260807 08:51:16.198407 30395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:16.198138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:42791 (local address 127.4.159.129:34607)
0807 08:51:16.198292 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:16.198328 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:42791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":77,"threads_started":1}
I20260807 08:51:17.075598 30398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:17.075391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:42791 (local address 127.4.159.131:43789)
0807 08:51:17.075514 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:17.075550 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:42791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":73,"threads_started":1}
I20260807 08:51:17.075610 30402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:17.075390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:42791 (local address 127.4.159.130:42747)
0807 08:51:17.075537 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:17.075565 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:42791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":75,"threads_started":1}
I20260807 08:51:17.198985 30404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:17.198799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:42791 (local address 127.4.159.129:33057)
0807 08:51:17.198906 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:17.198941 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:42791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":59,"threads_started":1}
I20260807 08:51:18.076205 30405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:18.075999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:42791 (local address 127.4.159.130:52855)
0807 08:51:18.076125 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:18.076159 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:42791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":71,"threads_started":1}
I20260807 08:51:18.076216 30406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:18.075997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:42791 (local address 127.4.159.131:53669)
0807 08:51:18.076146 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:18.076175 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:42791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":75,"threads_started":1}
I20260807 08:51:18.199568 30407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:18.199349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:42791 (local address 127.4.159.129:35449)
0807 08:51:18.199489 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:18.199523 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:42791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":79,"threads_started":1}
I20260807 08:51:19.076860 30408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:19.076649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:42791 (local address 127.4.159.130:53161)
0807 08:51:19.076779 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:19.076813 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:42791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":78,"threads_started":1}
I20260807 08:51:19.076869 30411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:19.076649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:42791 (local address 127.4.159.131:56559)
0807 08:51:19.076796 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:19.076827 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:42791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":93,"threads_started":1}
I20260807 08:51:19.200145 30415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:19.199933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:42791 (local address 127.4.159.129:42391)
0807 08:51:19.200068 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:19.200102 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:42791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":74,"threads_started":1}
I20260807 08:51:20.077443 30418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:20.077253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:42791 (local address 127.4.159.130:37779)
0807 08:51:20.077371 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:20.077402 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:42791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":59,"threads_started":1}
I20260807 08:51:20.077443 30417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:20.077231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:42791 (local address 127.4.159.131:53279)
0807 08:51:20.077364 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:20.077397 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:42791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":65,"threads_started":1}
I20260807 08:51:20.200716 30419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:20.200537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:42791 (local address 127.4.159.129:38833)
0807 08:51:20.200642 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:20.200672 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:42791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":57,"threads_started":1}
I20260807 08:51:21.078087 30421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:21.077879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:42791 (local address 127.4.159.130:40487)
0807 08:51:21.078005 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:21.078040 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:42791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":75,"threads_started":1}
I20260807 08:51:21.078105 30420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:21.077879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:42791 (local address 127.4.159.131:42107)
0807 08:51:21.078026 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:21.078053 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:42791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":77,"threads_started":1}
I20260807 08:51:21.201292 30422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:21.201058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:42791 (local address 127.4.159.129:42909)
0807 08:51:21.201210 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:21.201246 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:42791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":87,"threads_started":1}
I20260807 08:51:22.078677 30423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:22.078485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:42791 (local address 127.4.159.130:43463)
0807 08:51:22.078596 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:22.078626 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:42791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":60,"threads_started":1}
I20260807 08:51:22.078682 30425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:22.078501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:42791 (local address 127.4.159.131:34169)
0807 08:51:22.078612 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:22.078641 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:42791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":57,"threads_started":1}
I20260807 08:51:22.201938 30427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:22.201721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:42791 (local address 127.4.159.129:54281)
0807 08:51:22.201856 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:22.201892 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.190:42791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":77,"threads_started":1}
I20260807 08:51:22.452188 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 29651
I20260807 08:51:22.462654 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 29910
I20260807 08:51:22.469270 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 30122
2026-08-07T08:51:22Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58905 ms)
[----------] 30 tests from SecurityITest (228978 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786092453486433-4734-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:51:22 dist-test-slave-6zbq krb5kdc[30434](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:51:22 dist-test-slave-6zbq krb5kdc[30434](info): set up 2 sockets
Aug 07 08:51:22 dist-test-slave-6zbq krb5kdc[30434](info): commencing operation
krb5kdc: starting...
W20260807 08:51:23.585753 4734 mini_kdc.cc:121] Time spent starting KDC: real 1.095s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786092453486433-4734-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786092453486433-4734-0/krb5kdc/test-user.keytab.
Aug 07 08:51:23 dist-test-slave-6zbq krb5kdc[30434](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092683, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:51:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:51:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.159.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.159.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.159.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab.
Entry for principal kudu/127.4.159.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab.
I20260807 08:51:23.690709 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786092453486433-4734-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786092453486433-4734-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786092453486433-4734-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.159.190:46719
--webserver_interface=127.4.159.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab
--principal=kudu/127.4.159.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:34483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.159.190:46719
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:51:23.769690 30452 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:51:23.769881 30452 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:51:23.769910 30452 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:51:23.771031 30452 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:51:23.771070 30452 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:51:23.771087 30452 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:51:23.771103 30452 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:51:23.771118 30452 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:51:23.772541 30452 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:34483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786092453486433-4734-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786092453486433-4734-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.159.190:46719
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab
--principal=kudu/127.4.159.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.159.190:46719
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.159.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30452
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786092453486433-4734-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:51:23.772768 30452 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:51:23.772961 30452 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:51:23.777504 30459 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:51:23.777549 30461 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:51:23.777549 30469 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:51:23.777686 30452 server_base.cc:1034] running on GCE node
I20260807 08:51:23.777923 30452 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:51:23.778169 30452 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:51:23.779299 30452 hybrid_clock.cc:648] HybridClock initialized: now 1786092683779275 us; error 40 us; skew 500 ppm
Aug 07 08:51:23 dist-test-slave-6zbq krb5kdc[30434](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092683, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:51:23.785403 30452 init.cc:377] Logged in from keytab as kudu/127.4.159.190@KRBTEST.COM (short username kudu)
I20260807 08:51:23.785816 30452 webserver.cc:460] Webserver started at http://127.4.159.190:43803/ using document root <none> and password file <none>
I20260807 08:51:23.786022 30452 fs_manager.cc:359] Metadata directory not provided
I20260807 08:51:23.786069 30452 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:51:23.786190 30452 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:51:23.787194 30452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786092453486433-4734-0/minicluster-data/master-0/data/instance:
uuid: "b1552ebdcb5441e1a2789a5420a345ed"
format_stamp: "Formatted at 2026-08-07 08:51:23 on dist-test-slave-6zbq"
server_key: "ccd3e95032814bffab1df20d122bf25f"
server_key_iv: "ba2b75a6116f583062c912d7e9342f1d"
server_key_version: "encryptionkey@0"
I20260807 08:51:23.787515 30452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786092453486433-4734-0/minicluster-data/master-0/wal/instance:
uuid: "b1552ebdcb5441e1a2789a5420a345ed"
format_stamp: "Formatted at 2026-08-07 08:51:23 on dist-test-slave-6zbq"
server_key: "ccd3e95032814bffab1df20d122bf25f"
server_key_iv: "ba2b75a6116f583062c912d7e9342f1d"
server_key_version: "encryptionkey@0"
I20260807 08:51:23.788672 30452 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 08:51:23.789289 30480 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:51:23.789433 30452 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 08:51:23.789515 30452 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786092453486433-4734-0/minicluster-data/master-0/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786092453486433-4734-0/minicluster-data/master-0/wal
uuid: "b1552ebdcb5441e1a2789a5420a345ed"
format_stamp: "Formatted at 2026-08-07 08:51:23 on dist-test-slave-6zbq"
server_key: "ccd3e95032814bffab1df20d122bf25f"
server_key_iv: "ba2b75a6116f583062c912d7e9342f1d"
server_key_version: "encryptionkey@0"
I20260807 08:51:23.789579 30452 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786092453486433-4734-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786092453486433-4734-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786092453486433-4734-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:51:23.813552 30452 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:51:23.815500 30452 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:51:23.815629 30452 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:51:23.819416 30452 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.190:46719
I20260807 08:51:23.819455 30563 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.190:46719 every 8 connection(s)
I20260807 08:51:23.819762 30452 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
I20260807 08:51:23.820555 30564 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:51:23.822811 30564 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b1552ebdcb5441e1a2789a5420a345ed: Bootstrap starting.
I20260807 08:51:23.823361 30564 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b1552ebdcb5441e1a2789a5420a345ed: Neither blocks nor log segments found. Creating new log.
I20260807 08:51:23.823613 30564 log.cc:824] T 00000000000000000000000000000000 P b1552ebdcb5441e1a2789a5420a345ed: Log is configured to *not* fsync() on all Append() calls
I20260807 08:51:23.824234 30564 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b1552ebdcb5441e1a2789a5420a345ed: No bootstrap required, opened a new log
I20260807 08:51:23.824841 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 30452
I20260807 08:51:23.824947 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786092453486433-4734-0/minicluster-data/master-0/wal/instance
I20260807 08:51:23.825160 4734 external_mini_cluster.cc:1428] Setting key e6f9c37a18ab61d58137d8273801d875
I20260807 08:51:23.826328 30564 raft_consensus.cc:348] T 00000000000000000000000000000000 P b1552ebdcb5441e1a2789a5420a345ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1552ebdcb5441e1a2789a5420a345ed" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 46719 } }
I20260807 08:51:23.826491 30564 raft_consensus.cc:374] T 00000000000000000000000000000000 P b1552ebdcb5441e1a2789a5420a345ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:51:23.826514 30564 raft_consensus.cc:729] T 00000000000000000000000000000000 P b1552ebdcb5441e1a2789a5420a345ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1552ebdcb5441e1a2789a5420a345ed, State: Initialized, Role: FOLLOWER
I20260807 08:51:23.826656 30564 consensus_queue.cc:260] T 00000000000000000000000000000000 P b1552ebdcb5441e1a2789a5420a345ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1552ebdcb5441e1a2789a5420a345ed" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 46719 } }
I20260807 08:51:23.826745 30564 raft_consensus.cc:388] T 00000000000000000000000000000000 P b1552ebdcb5441e1a2789a5420a345ed [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:51:23.826817 30564 raft_consensus.cc:482] T 00000000000000000000000000000000 P b1552ebdcb5441e1a2789a5420a345ed [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:51:23.826870 30564 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b1552ebdcb5441e1a2789a5420a345ed [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:51:23.827510 30564 raft_consensus.cc:504] T 00000000000000000000000000000000 P b1552ebdcb5441e1a2789a5420a345ed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1552ebdcb5441e1a2789a5420a345ed" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 46719 } }
I20260807 08:51:23.827607 30564 leader_election.cc:302] T 00000000000000000000000000000000 P b1552ebdcb5441e1a2789a5420a345ed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b1552ebdcb5441e1a2789a5420a345ed; no voters:
I20260807 08:51:23.827777 30564 leader_election.cc:288] T 00000000000000000000000000000000 P b1552ebdcb5441e1a2789a5420a345ed [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:51:23.827807 30574 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b1552ebdcb5441e1a2789a5420a345ed [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:51:23.827953 30574 raft_consensus.cc:686] T 00000000000000000000000000000000 P b1552ebdcb5441e1a2789a5420a345ed [term 1 LEADER]: Becoming Leader. State: Replica: b1552ebdcb5441e1a2789a5420a345ed, State: Running, Role: LEADER
I20260807 08:51:23.828040 30564 sys_catalog.cc:564] T 00000000000000000000000000000000 P b1552ebdcb5441e1a2789a5420a345ed [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:51:23.828114 30574 consensus_queue.cc:237] T 00000000000000000000000000000000 P b1552ebdcb5441e1a2789a5420a345ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1552ebdcb5441e1a2789a5420a345ed" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 46719 } }
I20260807 08:51:23.828486 30574 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1552ebdcb5441e1a2789a5420a345ed [sys.catalog]: SysCatalogTable state changed. Reason: New leader b1552ebdcb5441e1a2789a5420a345ed. Latest consensus state: current_term: 1 leader_uuid: "b1552ebdcb5441e1a2789a5420a345ed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1552ebdcb5441e1a2789a5420a345ed" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 46719 } } }
I20260807 08:51:23.828537 30575 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1552ebdcb5441e1a2789a5420a345ed [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b1552ebdcb5441e1a2789a5420a345ed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1552ebdcb5441e1a2789a5420a345ed" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 46719 } } }
I20260807 08:51:23.828655 30574 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1552ebdcb5441e1a2789a5420a345ed [sys.catalog]: This master's current role is: LEADER
I20260807 08:51:23.828663 30575 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1552ebdcb5441e1a2789a5420a345ed [sys.catalog]: This master's current role is: LEADER
I20260807 08:51:23.828881 30585 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:51:23.829425 30585 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:51:23.831015 30585 catalog_manager.cc:1347] Generated new cluster ID: fac9b820b0984dbd8081b954e87c4519
I20260807 08:51:23.831067 30585 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 08:51:23 dist-test-slave-6zbq krb5kdc[30434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092683, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.159.190@KRBTEST.COM
I20260807 08:51:23.835615 30573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:23.825632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51162 (local address 127.4.159.190:46719)
0807 08:51:23.825783 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:23.825792 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:23.825817 (+ 25us) server_negotiation.cc:407] Connection header received
0807 08:51:23.826500 (+ 683us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:23.826509 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:23.826573 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:23.826705 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:51:23.827454 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:23.828621 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:23.829661 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:23.830149 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:23.830257 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:23.832501 (+ 2244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:51:23.832505 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:51:23.832505 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:51:23.832517 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:51:23.834005 (+ 1488us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:51:23.834323 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:51:23.834326 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:51:23.834326 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:51:23.834465 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:51:23.834638 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:51:23.834639 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:51:23.834639 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:51:23.834859 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:51:23.835330 (+ 471us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:23.835388 (+ 58us) server_negotiation.cc:299] Negotiation successful
0807 08:51:23.835484 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":71,"threads_started":1}
I20260807 08:51:23.841270 30585 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:51:23.841929 30585 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:51:23.849191 30585 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b1552ebdcb5441e1a2789a5420a345ed: Generated new TSK 0
I20260807 08:51:23.849350 30585 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.159.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.159.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.159.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.129.keytab.
Entry for principal kudu/127.4.159.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.129.keytab.
I20260807 08:51:23.876190 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786092453486433-4734-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786092453486433-4734-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786092453486433-4734-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786092453486433-4734-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.159.129:0
--local_ip_for_outbound_sockets=127.4.159.129
--webserver_interface=127.4.159.129
--webserver_port=0
--tserver_master_addrs=127.4.159.190:46719
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.129.keytab
--principal=kudu/127.4.159.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:34483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:51:23.975726 30605 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:51:23.975927 30605 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:51:23.975960 30605 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:51:23.977064 30605 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:51:23.977101 30605 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:51:23.977136 30605 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.159.129
I20260807 08:51:23.978593 30605 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:34483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786092453486433-4734-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786092453486433-4734-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.129.keytab
--principal=kudu/127.4.159.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.159.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786092453486433-4734-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.159.129
--webserver_port=0
--tserver_master_addrs=127.4.159.190:46719
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30605
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.159.129
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786092453486433-4734-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:51:23.978826 30605 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:51:23.979022 30605 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:51:23.983769 30618 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:51:23.983837 30614 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:51:23.983952 30605 server_base.cc:1034] running on GCE node
W20260807 08:51:23.983754 30613 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:51:23.984220 30605 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:51:23.984452 30605 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:51:23.985605 30605 hybrid_clock.cc:648] HybridClock initialized: now 1786092683985588 us; error 28 us; skew 500 ppm
Aug 07 08:51:23 dist-test-slave-6zbq krb5kdc[30434](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092683, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:51:23.987950 30605 init.cc:377] Logged in from keytab as kudu/127.4.159.129@KRBTEST.COM (short username kudu)
I20260807 08:51:23.988355 30605 webserver.cc:460] Webserver started at http://127.4.159.129:39493/ using document root <none> and password file <none>
I20260807 08:51:23.988564 30605 fs_manager.cc:359] Metadata directory not provided
I20260807 08:51:23.988611 30605 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:51:23.988736 30605 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:51:23.989706 30605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786092453486433-4734-0/minicluster-data/ts-0/data/instance:
uuid: "a3bd5d22c741407d94cb6e145688dc38"
format_stamp: "Formatted at 2026-08-07 08:51:23 on dist-test-slave-6zbq"
server_key: "7b21b8b421c096ae185933408e7fd500"
server_key_iv: "4591da40b469c272fa2c719c97d07784"
server_key_version: "encryptionkey@0"
I20260807 08:51:23.990028 30605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786092453486433-4734-0/minicluster-data/ts-0/wal/instance:
uuid: "a3bd5d22c741407d94cb6e145688dc38"
format_stamp: "Formatted at 2026-08-07 08:51:23 on dist-test-slave-6zbq"
server_key: "7b21b8b421c096ae185933408e7fd500"
server_key_iv: "4591da40b469c272fa2c719c97d07784"
server_key_version: "encryptionkey@0"
I20260807 08:51:23.991294 30605 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 08:51:23.991983 30636 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:51:23.992146 30605 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 08:51:23.992233 30605 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786092453486433-4734-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786092453486433-4734-0/minicluster-data/ts-0/wal
uuid: "a3bd5d22c741407d94cb6e145688dc38"
format_stamp: "Formatted at 2026-08-07 08:51:23 on dist-test-slave-6zbq"
server_key: "7b21b8b421c096ae185933408e7fd500"
server_key_iv: "4591da40b469c272fa2c719c97d07784"
server_key_version: "encryptionkey@0"
I20260807 08:51:23.992300 30605 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786092453486433-4734-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786092453486433-4734-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786092453486433-4734-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:51:24.003420 30605 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:51:24.005002 30605 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:51:24.005127 30605 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:51:24.005321 30605 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:51:24.005580 30605 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:51:24.005612 30605 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:51:24.005638 30605 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:51:24.005656 30605 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:51:24.011111 30605 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.129:42435
I20260807 08:51:24.011174 30807 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.129:42435 every 8 connection(s)
I20260807 08:51:24.011443 30605 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786092453486433-4734-0/minicluster-data/ts-0/data/info.pb
Aug 07 08:51:24 dist-test-slave-6zbq krb5kdc[30434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092683, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.129@KRBTEST.COM for kudu/127.4.159.190@KRBTEST.COM
I20260807 08:51:24.019771 30573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:24.012125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.129:48981 (local address 127.4.159.190:46719)
0807 08:51:24.012185 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:24.012187 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:24.012370 (+ 183us) server_negotiation.cc:407] Connection header received
0807 08:51:24.012815 (+ 445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:24.012816 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:24.012826 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:24.012877 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:51:24.013723 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:24.014242 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:24.015185 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:24.015375 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:24.015424 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:24.017236 (+ 1812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:51:24.017237 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:51:24.017238 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:51:24.017246 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:51:24.018566 (+ 1320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:51:24.018917 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:51:24.018918 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:51:24.018918 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:51:24.018968 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:51:24.019172 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:51:24.019173 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:51:24.019174 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:51:24.019281 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:51:24.019337 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:24.019577 (+ 240us) server_negotiation.cc:299] Negotiation successful
0807 08:51:24.019667 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260807 08:51:24.019891 30812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:24.012081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:46719 (local address 127.4.159.129:48981)
0807 08:51:24.012253 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:24.012294 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 08:51:24.012741 (+ 447us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:51:24.012942 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:51:24.012945 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:51:24.013469 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:51:24.013646 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:24.013651 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:24.014303 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:24.014303 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:24.015116 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:24.015118 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:24.015431 (+ 313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:24.015432 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:24.015527 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:24.015899 (+ 372us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:51:24.015907 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:51:24.017147 (+ 1240us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:51:24.018636 (+ 1489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:51:24.018643 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:51:24.018645 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:51:24.018847 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:51:24.019020 (+ 173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:51:24.019021 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:51:24.019021 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:51:24.019120 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:51:24.019335 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:51:24.019342 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:51:24.019466 (+ 124us) client_negotiation.cc:769] Sending connection context
0807 08:51:24.019550 (+ 84us) client_negotiation.cc:240] Negotiation successful
0807 08:51:24.019694 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":90,"threads_started":1}
I20260807 08:51:24.020364 30809 heartbeater.cc:347] Connected to a master server at 127.4.159.190:46719
I20260807 08:51:24.020452 30809 heartbeater.cc:464] Registering TS with master...
I20260807 08:51:24.020612 30809 heartbeater.cc:511] Master 127.4.159.190:46719 requested a full tablet report, sending...
I20260807 08:51:24.021142 30507 ts_manager.cc:194] Registered new tserver with Master: a3bd5d22c741407d94cb6e145688dc38 (127.4.159.129:42435)
I20260807 08:51:24.021217 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 30605
I20260807 08:51:24.021322 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786092453486433-4734-0/minicluster-data/ts-0/wal/instance
I20260807 08:51:24.021525 4734 external_mini_cluster.cc:1428] Setting key 510b929e0beabc843273196aa455ff2a
I20260807 08:51:24.022042 30507 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.159.129@KRBTEST.COM'} at 127.4.159.129:48981
WARNING: no policy specified for kudu/127.4.159.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.159.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.159.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.130.keytab.
Entry for principal kudu/127.4.159.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.130.keytab.
I20260807 08:51:24.044021 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786092453486433-4734-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786092453486433-4734-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786092453486433-4734-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786092453486433-4734-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.159.130:0
--local_ip_for_outbound_sockets=127.4.159.130
--webserver_interface=127.4.159.130
--webserver_port=0
--tserver_master_addrs=127.4.159.190:46719
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.130.keytab
--principal=kudu/127.4.159.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:34483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:51:24.123018 30816 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:51:24.123190 30816 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:51:24.123211 30816 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:51:24.124256 30816 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:51:24.124289 30816 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:51:24.124313 30816 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.159.130
I20260807 08:51:24.125662 30816 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:34483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786092453486433-4734-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786092453486433-4734-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.130.keytab
--principal=kudu/127.4.159.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.159.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786092453486433-4734-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.159.130
--webserver_port=0
--tserver_master_addrs=127.4.159.190:46719
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30816
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.159.130
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786092453486433-4734-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:51:24.125840 30816 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:51:24.126008 30816 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:51:24.130652 30826 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:51:24.130697 30830 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:51:24.130755 30816 server_base.cc:1034] running on GCE node
W20260807 08:51:24.130697 30827 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:51:24.131004 30816 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:51:24.131228 30816 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:51:24.132359 30816 hybrid_clock.cc:648] HybridClock initialized: now 1786092684132345 us; error 26 us; skew 500 ppm
Aug 07 08:51:24 dist-test-slave-6zbq krb5kdc[30434](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092684, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:51:24.134617 30816 init.cc:377] Logged in from keytab as kudu/127.4.159.130@KRBTEST.COM (short username kudu)
I20260807 08:51:24.135004 30816 webserver.cc:460] Webserver started at http://127.4.159.130:36805/ using document root <none> and password file <none>
I20260807 08:51:24.135183 30816 fs_manager.cc:359] Metadata directory not provided
I20260807 08:51:24.135221 30816 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:51:24.135309 30816 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:51:24.136178 30816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786092453486433-4734-0/minicluster-data/ts-1/data/instance:
uuid: "c21b7be4cc70482b951e449d84955f05"
format_stamp: "Formatted at 2026-08-07 08:51:24 on dist-test-slave-6zbq"
server_key: "5a01c694176c30bb7ba615e38000cd18"
server_key_iv: "a91abd37d95718ab7363ba4b9bb23c83"
server_key_version: "encryptionkey@0"
I20260807 08:51:24.136446 30816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786092453486433-4734-0/minicluster-data/ts-1/wal/instance:
uuid: "c21b7be4cc70482b951e449d84955f05"
format_stamp: "Formatted at 2026-08-07 08:51:24 on dist-test-slave-6zbq"
server_key: "5a01c694176c30bb7ba615e38000cd18"
server_key_iv: "a91abd37d95718ab7363ba4b9bb23c83"
server_key_version: "encryptionkey@0"
I20260807 08:51:24.137532 30816 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260807 08:51:24.138226 30841 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:51:24.138386 30816 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 08:51:24.138461 30816 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786092453486433-4734-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786092453486433-4734-0/minicluster-data/ts-1/wal
uuid: "c21b7be4cc70482b951e449d84955f05"
format_stamp: "Formatted at 2026-08-07 08:51:24 on dist-test-slave-6zbq"
server_key: "5a01c694176c30bb7ba615e38000cd18"
server_key_iv: "a91abd37d95718ab7363ba4b9bb23c83"
server_key_version: "encryptionkey@0"
I20260807 08:51:24.138525 30816 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786092453486433-4734-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786092453486433-4734-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786092453486433-4734-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:51:24.153170 30816 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:51:24.155134 30816 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:51:24.155259 30816 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:51:24.155449 30816 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:51:24.155715 30816 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:51:24.155752 30816 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:51:24.155773 30816 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:51:24.155797 30816 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:51:24.161156 30816 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.130:42893
I20260807 08:51:24.161201 31014 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.130:42893 every 8 connection(s)
I20260807 08:51:24.161448 30816 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786092453486433-4734-0/minicluster-data/ts-1/data/info.pb
Aug 07 08:51:24 dist-test-slave-6zbq krb5kdc[30434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092684, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.130@KRBTEST.COM for kudu/127.4.159.190@KRBTEST.COM
I20260807 08:51:24.168054 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 30816
I20260807 08:51:24.168174 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786092453486433-4734-0/minicluster-data/ts-1/wal/instance
I20260807 08:51:24.168380 4734 external_mini_cluster.cc:1428] Setting key 702becbe3d461a91518c3fc9aa2ae732
I20260807 08:51:24.169739 30573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:24.162034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.130:45103 (local address 127.4.159.190:46719)
0807 08:51:24.162095 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:24.162098 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:24.162244 (+ 146us) server_negotiation.cc:407] Connection header received
0807 08:51:24.162724 (+ 480us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:24.162725 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:24.162735 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:24.162788 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:51:24.163577 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:24.164060 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:24.164958 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:24.165144 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:24.165192 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:24.167254 (+ 2062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:51:24.167255 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:51:24.167255 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:51:24.167261 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:51:24.168461 (+ 1200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:51:24.168878 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:51:24.168878 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:51:24.168879 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:51:24.168935 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:51:24.169166 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:51:24.169166 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:51:24.169167 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:51:24.169300 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:51:24.169361 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:24.169562 (+ 201us) server_negotiation.cc:299] Negotiation successful
0807 08:51:24.169641 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260807 08:51:24.169796 31018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:24.162003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:46719 (local address 127.4.159.130:45103)
0807 08:51:24.162146 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:24.162180 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0807 08:51:24.162633 (+ 453us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:51:24.162844 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:51:24.162847 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:51:24.163344 (+ 497us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:51:24.163514 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:24.163518 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:24.164119 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:24.164119 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:24.164897 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:24.164899 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:24.165200 (+ 301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:24.165201 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:24.165294 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:24.165668 (+ 374us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:51:24.165676 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:51:24.167178 (+ 1502us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:51:24.168529 (+ 1351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:51:24.168536 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:51:24.168539 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:51:24.168820 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:51:24.168982 (+ 162us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:51:24.168983 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:51:24.168983 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:51:24.169107 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:51:24.169361 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:51:24.169364 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:51:24.169463 (+ 99us) client_negotiation.cc:769] Sending connection context
0807 08:51:24.169532 (+ 69us) client_negotiation.cc:240] Negotiation successful
0807 08:51:24.169638 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":80,"threads_started":1}
I20260807 08:51:24.170166 31016 heartbeater.cc:347] Connected to a master server at 127.4.159.190:46719
I20260807 08:51:24.170254 31016 heartbeater.cc:464] Registering TS with master...
I20260807 08:51:24.170435 31016 heartbeater.cc:511] Master 127.4.159.190:46719 requested a full tablet report, sending...
I20260807 08:51:24.170787 30507 ts_manager.cc:194] Registered new tserver with Master: c21b7be4cc70482b951e449d84955f05 (127.4.159.130:42893)
I20260807 08:51:24.171137 30507 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.159.130@KRBTEST.COM'} at 127.4.159.130:45103
WARNING: no policy specified for kudu/127.4.159.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.159.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.159.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.131.keytab.
Entry for principal kudu/127.4.159.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.131.keytab.
I20260807 08:51:24.190460 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786092453486433-4734-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786092453486433-4734-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786092453486433-4734-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786092453486433-4734-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.159.131:0
--local_ip_for_outbound_sockets=127.4.159.131
--webserver_interface=127.4.159.131
--webserver_port=0
--tserver_master_addrs=127.4.159.190:46719
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.131.keytab
--principal=kudu/127.4.159.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:34483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:51:24.268129 31021 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:51:24.268312 31021 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:51:24.268333 31021 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:51:24.269381 31021 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:51:24.269412 31021 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:51:24.269436 31021 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.159.131
I20260807 08:51:24.270803 31021 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:34483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786092453486433-4734-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786092453486433-4734-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.131.keytab
--principal=kudu/127.4.159.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.159.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786092453486433-4734-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.159.131
--webserver_port=0
--tserver_master_addrs=127.4.159.190:46719
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31021
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.159.131
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786092453486433-4734-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:51:24.270967 31021 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:51:24.271136 31021 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:51:24.275840 31029 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:51:24.275837 31032 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:51:24.275915 31021 server_base.cc:1034] running on GCE node
W20260807 08:51:24.275843 31028 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:51:24.276221 31021 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:51:24.276454 31021 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:51:24.277585 31021 hybrid_clock.cc:648] HybridClock initialized: now 1786092684277574 us; error 28 us; skew 500 ppm
Aug 07 08:51:24 dist-test-slave-6zbq krb5kdc[30434](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092684, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:51:24.279886 31021 init.cc:377] Logged in from keytab as kudu/127.4.159.131@KRBTEST.COM (short username kudu)
I20260807 08:51:24.280279 31021 webserver.cc:460] Webserver started at http://127.4.159.131:46451/ using document root <none> and password file <none>
I20260807 08:51:24.280463 31021 fs_manager.cc:359] Metadata directory not provided
I20260807 08:51:24.280504 31021 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:51:24.280625 31021 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:51:24.281605 31021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786092453486433-4734-0/minicluster-data/ts-2/data/instance:
uuid: "90c0cbd4946a4ed9b7308742c5bd12ce"
format_stamp: "Formatted at 2026-08-07 08:51:24 on dist-test-slave-6zbq"
server_key: "50281e8ac1f785fc12710f477cf70370"
server_key_iv: "96f3d5e0d63a2303f239a42667328260"
server_key_version: "encryptionkey@0"
I20260807 08:51:24.281921 31021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786092453486433-4734-0/minicluster-data/ts-2/wal/instance:
uuid: "90c0cbd4946a4ed9b7308742c5bd12ce"
format_stamp: "Formatted at 2026-08-07 08:51:24 on dist-test-slave-6zbq"
server_key: "50281e8ac1f785fc12710f477cf70370"
server_key_iv: "96f3d5e0d63a2303f239a42667328260"
server_key_version: "encryptionkey@0"
I20260807 08:51:24.283208 31021 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 08:51:24.283896 31039 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:51:24.284039 31021 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 08:51:24.284118 31021 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786092453486433-4734-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786092453486433-4734-0/minicluster-data/ts-2/wal
uuid: "90c0cbd4946a4ed9b7308742c5bd12ce"
format_stamp: "Formatted at 2026-08-07 08:51:24 on dist-test-slave-6zbq"
server_key: "50281e8ac1f785fc12710f477cf70370"
server_key_iv: "96f3d5e0d63a2303f239a42667328260"
server_key_version: "encryptionkey@0"
I20260807 08:51:24.284183 31021 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786092453486433-4734-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786092453486433-4734-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786092453486433-4734-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:51:24.297519 31021 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:51:24.299594 31021 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:51:24.299722 31021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:51:24.299921 31021 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:51:24.300199 31021 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:51:24.300231 31021 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:51:24.300252 31021 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:51:24.300285 31021 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:51:24.305711 31021 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.131:44427
I20260807 08:51:24.305752 31243 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.131:44427 every 8 connection(s)
I20260807 08:51:24.306046 31021 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786092453486433-4734-0/minicluster-data/ts-2/data/info.pb
Aug 07 08:51:24 dist-test-slave-6zbq krb5kdc[30434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092684, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.131@KRBTEST.COM for kudu/127.4.159.190@KRBTEST.COM
I20260807 08:51:24.313828 30573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:24.306742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.131:35815 (local address 127.4.159.190:46719)
0807 08:51:24.306803 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:24.306805 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:24.306959 (+ 154us) server_negotiation.cc:407] Connection header received
0807 08:51:24.307420 (+ 461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:24.307421 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:24.307430 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:24.307481 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:51:24.308340 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:24.308821 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:24.309697 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:24.309868 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:24.309909 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:24.311779 (+ 1870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:51:24.311780 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:51:24.311781 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:51:24.311787 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:51:24.312815 (+ 1028us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:51:24.313128 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:51:24.313129 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:51:24.313129 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:51:24.313183 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:51:24.313364 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:51:24.313365 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:51:24.313365 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:51:24.313463 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:51:24.313517 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:24.313711 (+ 194us) server_negotiation.cc:299] Negotiation successful
0807 08:51:24.313766 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260807 08:51:24.313998 31246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:24.306686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:46719 (local address 127.4.159.131:35815)
0807 08:51:24.306841 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:24.306882 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 08:51:24.307345 (+ 463us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:51:24.307539 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:51:24.307542 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:51:24.308080 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:51:24.308273 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:24.308278 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:24.308880 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:24.308881 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:24.309637 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:24.309639 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:24.309920 (+ 281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:24.309921 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:24.310011 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:24.310458 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:51:24.310467 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:51:24.311696 (+ 1229us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:51:24.312876 (+ 1180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:51:24.312881 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:51:24.312882 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:51:24.313073 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:51:24.313227 (+ 154us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:51:24.313227 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:51:24.313227 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:51:24.313321 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:51:24.313513 (+ 192us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:51:24.313516 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:51:24.313618 (+ 102us) client_negotiation.cc:769] Sending connection context
0807 08:51:24.313693 (+ 75us) client_negotiation.cc:240] Negotiation successful
0807 08:51:24.313805 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":86,"threads_started":1}
I20260807 08:51:24.314466 31244 heartbeater.cc:347] Connected to a master server at 127.4.159.190:46719
I20260807 08:51:24.314550 31244 heartbeater.cc:464] Registering TS with master...
I20260807 08:51:24.314713 31244 heartbeater.cc:511] Master 127.4.159.190:46719 requested a full tablet report, sending...
I20260807 08:51:24.314772 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 31021
I20260807 08:51:24.314873 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786092453486433-4734-0/minicluster-data/ts-2/wal/instance
I20260807 08:51:24.315059 4734 external_mini_cluster.cc:1428] Setting key 7a0234a0ebddafd6385b256d56dd295a
I20260807 08:51:24.315078 30507 ts_manager.cc:194] Registered new tserver with Master: 90c0cbd4946a4ed9b7308742c5bd12ce (127.4.159.131:44427)
I20260807 08:51:24.315500 30507 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.159.131@KRBTEST.COM'} at 127.4.159.131:35815
I20260807 08:51:24.315897 4734 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 08:51:24.322013 30573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:24.317206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51166 (local address 127.4.159.190:46719)
0807 08:51:24.317251 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:24.317253 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:24.317327 (+ 74us) server_negotiation.cc:407] Connection header received
0807 08:51:24.317371 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:24.317372 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:24.317378 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:24.317422 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:51:24.317899 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:24.318366 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:24.319029 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:24.319196 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:24.319235 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:24.320214 (+ 979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:51:24.320214 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:51:24.320215 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:51:24.320219 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:51:24.321221 (+ 1002us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:51:24.321489 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:51:24.321489 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:51:24.321490 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:51:24.321528 (+ 38us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:51:24.321648 (+ 120us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:51:24.321648 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:51:24.321648 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:51:24.321770 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:51:24.321835 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:24.321906 (+ 71us) server_negotiation.cc:299] Negotiation successful
0807 08:51:24.321958 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260807 08:51:24.323298 30504 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51166:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 08:51:24.324746 30504 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:51:24.331023 31261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:24.326883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49194 (local address 127.4.159.131:44427)
0807 08:51:24.327031 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:24.327035 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:24.327454 (+ 419us) server_negotiation.cc:407] Connection header received
0807 08:51:24.327496 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:24.327499 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:24.327513 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:24.327578 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:51:24.327813 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:24.328817 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:24.329988 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:24.330635 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:24.330704 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:24.330758 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:24.330913 (+ 155us) server_negotiation.cc:299] Negotiation successful
0807 08:51:24.330951 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":78,"threads_started":1}
I20260807 08:51:24.331030 31259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:24.326648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.130:42893 (local address 127.0.0.1:56520)
0807 08:51:24.326765 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:24.326803 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 08:51:24.327027 (+ 224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:51:24.327274 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:51:24.327277 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:51:24.327279 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:51:24.327343 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:24.327345 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:24.328825 (+ 1480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:24.328825 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:24.330142 (+ 1317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:24.330144 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:24.330849 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:24.330850 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:24.330903 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:24.330906 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 08:51:24.330933 (+ 27us) client_negotiation.cc:240] Negotiation successful
0807 08:51:24.330960 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96,"mutex_wait_us":40,"thread_start_us":73,"threads_started":1}
I20260807 08:51:24.331283 31256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:24.326657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.129:42435 (local address 127.0.0.1:48234)
0807 08:51:24.327221 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:24.327231 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 08:51:24.327284 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:51:24.327806 (+ 522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:51:24.327806 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:51:24.327807 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:51:24.327825 (+ 18us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:24.327826 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:24.328772 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:24.328773 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:24.330081 (+ 1308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:24.330085 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:24.331007 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:24.331008 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:24.331071 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:24.331075 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 08:51:24.331172 (+ 97us) client_negotiation.cc:240] Negotiation successful
0807 08:51:24.331216 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":550,"mutex_wait_us":34,"thread_start_us":52,"threads_started":1}
I20260807 08:51:24.331306 31262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:24.327044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48234 (local address 127.4.159.129:42435)
0807 08:51:24.327214 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:24.327218 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:24.327472 (+ 254us) server_negotiation.cc:407] Connection header received
0807 08:51:24.327518 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:24.327520 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:24.327535 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:24.327602 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:51:24.327885 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:24.328679 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:24.330165 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:24.330861 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:24.330921 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:24.330972 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:24.331190 (+ 218us) server_negotiation.cc:299] Negotiation successful
0807 08:51:24.331231 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":84,"threads_started":1}
I20260807 08:51:24.331487 31260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:24.326826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56520 (local address 127.4.159.130:42893)
0807 08:51:24.326996 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:24.327000 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:24.327013 (+ 13us) server_negotiation.cc:407] Connection header received
0807 08:51:24.327131 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:24.327133 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:24.327149 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:24.327274 (+ 125us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:51:24.327516 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:24.328708 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:24.330199 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:24.330745 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:24.331229 (+ 484us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:24.331296 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:24.331321 (+ 25us) server_negotiation.cc:299] Negotiation successful
0807 08:51:24.331410 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":88,"threads_started":1}
I20260807 08:51:24.331029 31257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:24.326660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.131:44427 (local address 127.0.0.1:49194)
0807 08:51:24.327348 (+ 688us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:24.327356 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0807 08:51:24.327400 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:51:24.327692 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:51:24.327693 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:51:24.327695 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:51:24.327741 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:24.327742 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:24.328891 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:24.328892 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:24.329918 (+ 1026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:24.329921 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:24.330733 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:24.330735 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:24.330811 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:24.330828 (+ 17us) client_negotiation.cc:769] Sending connection context
0807 08:51:24.330892 (+ 64us) client_negotiation.cc:240] Negotiation successful
0807 08:51:24.330945 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":679,"mutex_wait_us":21,"thread_start_us":53,"threads_started":1}
I20260807 08:51:24.332080 30707 tablet_service.cc:1467] Processing CreateTablet for tablet 3f0ff9e54c0f4e52b617d1e3da1e8925 (DEFAULT_TABLE table=test-table [id=f4b542c1f1fc4d07b47f79c209fc5232]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:51:24.332160 30912 tablet_service.cc:1467] Processing CreateTablet for tablet 3f0ff9e54c0f4e52b617d1e3da1e8925 (DEFAULT_TABLE table=test-table [id=f4b542c1f1fc4d07b47f79c209fc5232]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:51:24.332403 30707 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f0ff9e54c0f4e52b617d1e3da1e8925. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:51:24.332453 30912 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f0ff9e54c0f4e52b617d1e3da1e8925. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:51:24.334861 31263 tablet_bootstrap.cc:492] T 3f0ff9e54c0f4e52b617d1e3da1e8925 P c21b7be4cc70482b951e449d84955f05: Bootstrap starting.
I20260807 08:51:24.334785 31127 tablet_service.cc:1467] Processing CreateTablet for tablet 3f0ff9e54c0f4e52b617d1e3da1e8925 (DEFAULT_TABLE table=test-table [id=f4b542c1f1fc4d07b47f79c209fc5232]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:51:24.334986 31264 tablet_bootstrap.cc:492] T 3f0ff9e54c0f4e52b617d1e3da1e8925 P a3bd5d22c741407d94cb6e145688dc38: Bootstrap starting.
I20260807 08:51:24.335095 31127 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f0ff9e54c0f4e52b617d1e3da1e8925. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:51:24.335598 31263 tablet_bootstrap.cc:654] T 3f0ff9e54c0f4e52b617d1e3da1e8925 P c21b7be4cc70482b951e449d84955f05: Neither blocks nor log segments found. Creating new log.
I20260807 08:51:24.335681 31264 tablet_bootstrap.cc:654] T 3f0ff9e54c0f4e52b617d1e3da1e8925 P a3bd5d22c741407d94cb6e145688dc38: Neither blocks nor log segments found. Creating new log.
I20260807 08:51:24.336056 31263 log.cc:824] T 3f0ff9e54c0f4e52b617d1e3da1e8925 P c21b7be4cc70482b951e449d84955f05: Log is configured to *not* fsync() on all Append() calls
I20260807 08:51:24.336211 31264 log.cc:824] T 3f0ff9e54c0f4e52b617d1e3da1e8925 P a3bd5d22c741407d94cb6e145688dc38: Log is configured to *not* fsync() on all Append() calls
I20260807 08:51:24.336742 31263 tablet_bootstrap.cc:492] T 3f0ff9e54c0f4e52b617d1e3da1e8925 P c21b7be4cc70482b951e449d84955f05: No bootstrap required, opened a new log
I20260807 08:51:24.336799 31263 ts_tablet_manager.cc:1397] T 3f0ff9e54c0f4e52b617d1e3da1e8925 P c21b7be4cc70482b951e449d84955f05: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 08:51:24.336943 31264 tablet_bootstrap.cc:492] T 3f0ff9e54c0f4e52b617d1e3da1e8925 P a3bd5d22c741407d94cb6e145688dc38: No bootstrap required, opened a new log
I20260807 08:51:24.337015 31264 ts_tablet_manager.cc:1397] T 3f0ff9e54c0f4e52b617d1e3da1e8925 P a3bd5d22c741407d94cb6e145688dc38: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 08:51:24.338383 31263 raft_consensus.cc:348] T 3f0ff9e54c0f4e52b617d1e3da1e8925 P c21b7be4cc70482b951e449d84955f05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c21b7be4cc70482b951e449d84955f05" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 42893 } } peers { permanent_uuid: "a3bd5d22c741407d94cb6e145688dc38" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 42435 } } peers { permanent_uuid: "90c0cbd4946a4ed9b7308742c5bd12ce" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 44427 } }
I20260807 08:51:24.338526 31263 raft_consensus.cc:374] T 3f0ff9e54c0f4e52b617d1e3da1e8925 P c21b7be4cc70482b951e449d84955f05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:51:24.338558 31263 raft_consensus.cc:729] T 3f0ff9e54c0f4e52b617d1e3da1e8925 P c21b7be4cc70482b951e449d84955f05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c21b7be4cc70482b951e449d84955f05, State: Initialized, Role: FOLLOWER
I20260807 08:51:24.338676 31263 consensus_queue.cc:260] T 3f0ff9e54c0f4e52b617d1e3da1e8925 P c21b7be4cc70482b951e449d84955f05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c21b7be4cc70482b951e449d84955f05" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 42893 } } peers { permanent_uuid: "a3bd5d22c741407d94cb6e145688dc38" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 42435 } } peers { permanent_uuid: "90c0cbd4946a4ed9b7308742c5bd12ce" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 44427 } }
I20260807 08:51:24.338922 31263 ts_tablet_manager.cc:1428] T 3f0ff9e54c0f4e52b617d1e3da1e8925 P c21b7be4cc70482b951e449d84955f05: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 08:51:24.338949 31016 heartbeater.cc:503] Master 127.4.159.190:46719 was elected leader, sending a full tablet report...
I20260807 08:51:24.338861 31264 raft_consensus.cc:348] T 3f0ff9e54c0f4e52b617d1e3da1e8925 P a3bd5d22c741407d94cb6e145688dc38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c21b7be4cc70482b951e449d84955f05" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 42893 } } peers { permanent_uuid: "a3bd5d22c741407d94cb6e145688dc38" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 42435 } } peers { permanent_uuid: "90c0cbd4946a4ed9b7308742c5bd12ce" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 44427 } }
I20260807 08:51:24.339015 31264 raft_consensus.cc:374] T 3f0ff9e54c0f4e52b617d1e3da1e8925 P a3bd5d22c741407d94cb6e145688dc38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:51:24.339048 31264 raft_consensus.cc:729] T 3f0ff9e54c0f4e52b617d1e3da1e8925 P a3bd5d22c741407d94cb6e145688dc38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3bd5d22c741407d94cb6e145688dc38, State: Initialized, Role: FOLLOWER
I20260807 08:51:24.339148 31264 consensus_queue.cc:260] T 3f0ff9e54c0f4e52b617d1e3da1e8925 P a3bd5d22c741407d94cb6e145688dc38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c21b7be4cc70482b951e449d84955f05" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 42893 } } peers { permanent_uuid: "a3bd5d22c741407d94cb6e145688dc38" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 42435 } } peers { permanent_uuid: "90c0cbd4946a4ed9b7308742c5bd12ce" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 44427 } }
I20260807 08:51:24.339406 31264 ts_tablet_manager.cc:1428] T 3f0ff9e54c0f4e52b617d1e3da1e8925 P a3bd5d22c741407d94cb6e145688dc38: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260807 08:51:24.339469 30809 heartbeater.cc:503] Master 127.4.159.190:46719 was elected leader, sending a full tablet report...
I20260807 08:51:24.339583 31267 tablet_bootstrap.cc:492] T 3f0ff9e54c0f4e52b617d1e3da1e8925 P 90c0cbd4946a4ed9b7308742c5bd12ce: Bootstrap starting.
I20260807 08:51:24.340178 31267 tablet_bootstrap.cc:654] T 3f0ff9e54c0f4e52b617d1e3da1e8925 P 90c0cbd4946a4ed9b7308742c5bd12ce: Neither blocks nor log segments found. Creating new log.
I20260807 08:51:24.340657 31267 log.cc:824] T 3f0ff9e54c0f4e52b617d1e3da1e8925 P 90c0cbd4946a4ed9b7308742c5bd12ce: Log is configured to *not* fsync() on all Append() calls
I20260807 08:51:24.341287 31267 tablet_bootstrap.cc:492] T 3f0ff9e54c0f4e52b617d1e3da1e8925 P 90c0cbd4946a4ed9b7308742c5bd12ce: No bootstrap required, opened a new log
I20260807 08:51:24.341341 31267 ts_tablet_manager.cc:1397] T 3f0ff9e54c0f4e52b617d1e3da1e8925 P 90c0cbd4946a4ed9b7308742c5bd12ce: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 08:51:24.342813 31267 raft_consensus.cc:348] T 3f0ff9e54c0f4e52b617d1e3da1e8925 P 90c0cbd4946a4ed9b7308742c5bd12ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c21b7be4cc70482b951e449d84955f05" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 42893 } } peers { permanent_uuid: "a3bd5d22c741407d94cb6e145688dc38" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 42435 } } peers { permanent_uuid: "90c0cbd4946a4ed9b7308742c5bd12ce" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 44427 } }
I20260807 08:51:24.342937 31267 raft_consensus.cc:374] T 3f0ff9e54c0f4e52b617d1e3da1e8925 P 90c0cbd4946a4ed9b7308742c5bd12ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:51:24.342958 31267 raft_consensus.cc:729] T 3f0ff9e54c0f4e52b617d1e3da1e8925 P 90c0cbd4946a4ed9b7308742c5bd12ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90c0cbd4946a4ed9b7308742c5bd12ce, State: Initialized, Role: FOLLOWER
I20260807 08:51:24.343029 31267 consensus_queue.cc:260] T 3f0ff9e54c0f4e52b617d1e3da1e8925 P 90c0cbd4946a4ed9b7308742c5bd12ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c21b7be4cc70482b951e449d84955f05" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 42893 } } peers { permanent_uuid: "a3bd5d22c741407d94cb6e145688dc38" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 42435 } } peers { permanent_uuid: "90c0cbd4946a4ed9b7308742c5bd12ce" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 44427 } }
I20260807 08:51:24.343219 31267 ts_tablet_manager.cc:1428] T 3f0ff9e54c0f4e52b617d1e3da1e8925 P 90c0cbd4946a4ed9b7308742c5bd12ce: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 08:51:24.343263 31244 heartbeater.cc:503] Master 127.4.159.190:46719 was elected leader, sending a full tablet report...
I20260807 08:51:24.409204 31268 raft_consensus.cc:482] T 3f0ff9e54c0f4e52b617d1e3da1e8925 P c21b7be4cc70482b951e449d84955f05 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:51:24.409322 31268 raft_consensus.cc:504] T 3f0ff9e54c0f4e52b617d1e3da1e8925 P c21b7be4cc70482b951e449d84955f05 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c21b7be4cc70482b951e449d84955f05" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 42893 } } peers { permanent_uuid: "a3bd5d22c741407d94cb6e145688dc38" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 42435 } } peers { permanent_uuid: "90c0cbd4946a4ed9b7308742c5bd12ce" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 44427 } }
I20260807 08:51:24.409696 31268 leader_election.cc:288] T 3f0ff9e54c0f4e52b617d1e3da1e8925 P c21b7be4cc70482b951e449d84955f05 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3bd5d22c741407d94cb6e145688dc38 (127.4.159.129:42435), 90c0cbd4946a4ed9b7308742c5bd12ce (127.4.159.131:44427)
W20260807 08:51:24.411929 31017 tablet.cc:2369] T 3f0ff9e54c0f4e52b617d1e3da1e8925 P c21b7be4cc70482b951e449d84955f05: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 08:51:24.412454 31018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:24.409855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.131:44427 (local address 127.4.159.130:36755)
0807 08:51:24.409953 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:24.409965 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 08:51:24.410020 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:51:24.410145 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:51:24.410146 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:51:24.410148 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:51:24.410225 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:24.410228 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:24.410902 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:24.410903 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:24.411634 (+ 731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:24.411637 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:24.412220 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:24.412221 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:24.412309 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:24.412313 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 08:51:24.412360 (+ 47us) client_negotiation.cc:240] Negotiation successful
0807 08:51:24.412403 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":78}
I20260807 08:51:24.412482 31261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:24.409931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.130:36755 (local address 127.4.159.131:44427)
0807 08:51:24.409969 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:24.409971 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:24.410010 (+ 39us) server_negotiation.cc:407] Connection header received
0807 08:51:24.410072 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:24.410073 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:24.410082 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:24.410136 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:51:24.410300 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:24.410826 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:24.411717 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:24.412148 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:24.412203 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:24.412212 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:24.412378 (+ 166us) server_negotiation.cc:299] Negotiation successful
0807 08:51:24.412416 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 08:51:24.412597 31274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:24.409859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.129:42435 (local address 127.4.159.130:46159)
0807 08:51:24.410035 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:24.410045 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 08:51:24.410132 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:51:24.410255 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:51:24.410256 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:51:24.410257 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:51:24.410357 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:24.410359 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:24.410936 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:24.410937 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:24.411676 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:24.411678 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:24.412363 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:24.412364 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:24.412435 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:24.412440 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 08:51:24.412491 (+ 51us) client_negotiation.cc:240] Negotiation successful
0807 08:51:24.412527 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"mutex_wait_us":13,"thread_start_us":85,"threads_started":1}
I20260807 08:51:24.412607 31262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:24.409888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.130:46159 (local address 127.4.159.129:42435)
0807 08:51:24.409951 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:24.409953 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:24.410111 (+ 158us) server_negotiation.cc:407] Connection header received
0807 08:51:24.410185 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:24.410186 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:24.410195 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:24.410249 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:51:24.410409 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:24.410858 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:24.411764 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:24.412277 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:24.412351 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:24.412359 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:24.412509 (+ 150us) server_negotiation.cc:299] Negotiation successful
0807 08:51:24.412548 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260807 08:51:24.412752 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f0ff9e54c0f4e52b617d1e3da1e8925" candidate_uuid: "c21b7be4cc70482b951e449d84955f05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90c0cbd4946a4ed9b7308742c5bd12ce" is_pre_election: true
I20260807 08:51:24.412885 31172 raft_consensus.cc:2454] T 3f0ff9e54c0f4e52b617d1e3da1e8925 P 90c0cbd4946a4ed9b7308742c5bd12ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c21b7be4cc70482b951e449d84955f05 in term 0.
I20260807 08:51:24.412973 30738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f0ff9e54c0f4e52b617d1e3da1e8925" candidate_uuid: "c21b7be4cc70482b951e449d84955f05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3bd5d22c741407d94cb6e145688dc38" is_pre_election: true
I20260807 08:51:24.413084 30843 leader_election.cc:302] T 3f0ff9e54c0f4e52b617d1e3da1e8925 P c21b7be4cc70482b951e449d84955f05 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90c0cbd4946a4ed9b7308742c5bd12ce, c21b7be4cc70482b951e449d84955f05; no voters:
I20260807 08:51:24.413130 30738 raft_consensus.cc:2454] T 3f0ff9e54c0f4e52b617d1e3da1e8925 P a3bd5d22c741407d94cb6e145688dc38 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c21b7be4cc70482b951e449d84955f05 in term 0.
I20260807 08:51:24.413198 31268 raft_consensus.cc:2781] T 3f0ff9e54c0f4e52b617d1e3da1e8925 P c21b7be4cc70482b951e449d84955f05 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:51:24.413252 31268 raft_consensus.cc:482] T 3f0ff9e54c0f4e52b617d1e3da1e8925 P c21b7be4cc70482b951e449d84955f05 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:51:24.413285 31268 raft_consensus.cc:3037] T 3f0ff9e54c0f4e52b617d1e3da1e8925 P c21b7be4cc70482b951e449d84955f05 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:51:24.413949 31268 raft_consensus.cc:504] T 3f0ff9e54c0f4e52b617d1e3da1e8925 P c21b7be4cc70482b951e449d84955f05 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c21b7be4cc70482b951e449d84955f05" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 42893 } } peers { permanent_uuid: "a3bd5d22c741407d94cb6e145688dc38" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 42435 } } peers { permanent_uuid: "90c0cbd4946a4ed9b7308742c5bd12ce" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 44427 } }
I20260807 08:51:24.414052 31268 leader_election.cc:288] T 3f0ff9e54c0f4e52b617d1e3da1e8925 P c21b7be4cc70482b951e449d84955f05 [CANDIDATE]: Term 1 election: Requested vote from peers a3bd5d22c741407d94cb6e145688dc38 (127.4.159.129:42435), 90c0cbd4946a4ed9b7308742c5bd12ce (127.4.159.131:44427)
I20260807 08:51:24.414244 30738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f0ff9e54c0f4e52b617d1e3da1e8925" candidate_uuid: "c21b7be4cc70482b951e449d84955f05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3bd5d22c741407d94cb6e145688dc38"
I20260807 08:51:24.414311 30738 raft_consensus.cc:3037] T 3f0ff9e54c0f4e52b617d1e3da1e8925 P a3bd5d22c741407d94cb6e145688dc38 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:51:24.414395 31172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f0ff9e54c0f4e52b617d1e3da1e8925" candidate_uuid: "c21b7be4cc70482b951e449d84955f05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90c0cbd4946a4ed9b7308742c5bd12ce"
I20260807 08:51:24.414495 31172 raft_consensus.cc:3037] T 3f0ff9e54c0f4e52b617d1e3da1e8925 P 90c0cbd4946a4ed9b7308742c5bd12ce [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:51:24.415032 30738 raft_consensus.cc:2454] T 3f0ff9e54c0f4e52b617d1e3da1e8925 P a3bd5d22c741407d94cb6e145688dc38 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c21b7be4cc70482b951e449d84955f05 in term 1.
I20260807 08:51:24.415217 30842 leader_election.cc:302] T 3f0ff9e54c0f4e52b617d1e3da1e8925 P c21b7be4cc70482b951e449d84955f05 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3bd5d22c741407d94cb6e145688dc38, c21b7be4cc70482b951e449d84955f05; no voters:
I20260807 08:51:24.415221 31172 raft_consensus.cc:2454] T 3f0ff9e54c0f4e52b617d1e3da1e8925 P 90c0cbd4946a4ed9b7308742c5bd12ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c21b7be4cc70482b951e449d84955f05 in term 1.
I20260807 08:51:24.415319 31268 raft_consensus.cc:2781] T 3f0ff9e54c0f4e52b617d1e3da1e8925 P c21b7be4cc70482b951e449d84955f05 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:51:24.415473 31268 raft_consensus.cc:686] T 3f0ff9e54c0f4e52b617d1e3da1e8925 P c21b7be4cc70482b951e449d84955f05 [term 1 LEADER]: Becoming Leader. State: Replica: c21b7be4cc70482b951e449d84955f05, State: Running, Role: LEADER
I20260807 08:51:24.415575 31268 consensus_queue.cc:237] T 3f0ff9e54c0f4e52b617d1e3da1e8925 P c21b7be4cc70482b951e449d84955f05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c21b7be4cc70482b951e449d84955f05" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 42893 } } peers { permanent_uuid: "a3bd5d22c741407d94cb6e145688dc38" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 42435 } } peers { permanent_uuid: "90c0cbd4946a4ed9b7308742c5bd12ce" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 44427 } }
I20260807 08:51:24.416339 30507 catalog_manager.cc:5495] T 3f0ff9e54c0f4e52b617d1e3da1e8925 P c21b7be4cc70482b951e449d84955f05 reported cstate change: term changed from 0 to 1, leader changed from <none> to c21b7be4cc70482b951e449d84955f05 (127.4.159.130). New cstate: current_term: 1 leader_uuid: "c21b7be4cc70482b951e449d84955f05" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c21b7be4cc70482b951e449d84955f05" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 42893 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a3bd5d22c741407d94cb6e145688dc38" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 42435 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90c0cbd4946a4ed9b7308742c5bd12ce" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 44427 } health_report { overall_health: UNKNOWN } } }
I20260807 08:51:24.443460 31260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:24.441242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56526 (local address 127.4.159.130:42893)
0807 08:51:24.441290 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:24.441291 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:24.441298 (+ 7us) server_negotiation.cc:407] Connection header received
0807 08:51:24.441361 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:24.441362 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:24.441375 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:24.441454 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:51:24.441580 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:24.442048 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:24.442782 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:24.442960 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:24.443012 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:24.443170 (+ 158us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:51:24.443261 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:51:24.443328 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:24.443362 (+ 34us) server_negotiation.cc:299] Negotiation successful
0807 08:51:24.443380 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260807 08:51:24.444947 30738 raft_consensus.cc:1261] T 3f0ff9e54c0f4e52b617d1e3da1e8925 P a3bd5d22c741407d94cb6e145688dc38 [term 1 FOLLOWER]: Refusing update from remote peer c21b7be4cc70482b951e449d84955f05: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:51:24.444959 31172 raft_consensus.cc:1261] T 3f0ff9e54c0f4e52b617d1e3da1e8925 P 90c0cbd4946a4ed9b7308742c5bd12ce [term 1 FOLLOWER]: Refusing update from remote peer c21b7be4cc70482b951e449d84955f05: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:51:24.445237 31278 consensus_queue.cc:1035] T 3f0ff9e54c0f4e52b617d1e3da1e8925 P c21b7be4cc70482b951e449d84955f05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90c0cbd4946a4ed9b7308742c5bd12ce" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 44427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:51:24.445369 31268 consensus_queue.cc:1035] T 3f0ff9e54c0f4e52b617d1e3da1e8925 P c21b7be4cc70482b951e449d84955f05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3bd5d22c741407d94cb6e145688dc38" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 42435 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:51:24.448112 31289 mvcc.cc:205] Tried to move back new op lower bound from 7315835635484348416 to 7315835635366699008. Current Snapshot: MvccSnapshot[applied={T|T < 7315835635484348416 or (T in {7315835635484348416})}]
I20260807 08:51:24.448243 31288 mvcc.cc:205] Tried to move back new op lower bound from 7315835635484348416 to 7315835635366699008. Current Snapshot: MvccSnapshot[applied={T|T < 7315835635484348416}]
I20260807 08:51:24.449576 30504 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51166:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:51:24.449656 30504 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51166:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:51:24.450291 30504 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b1552ebdcb5441e1a2789a5420a345ed: Sending DeleteTablet for 3 replicas of tablet 3f0ff9e54c0f4e52b617d1e3da1e8925
I20260807 08:51:24.450610 31127 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f0ff9e54c0f4e52b617d1e3da1e8925 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:51:24 UTC) from {username='kudu', principal='kudu/127.4.159.190@KRBTEST.COM'} at 127.0.0.1:49194
I20260807 08:51:24.450583 30912 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f0ff9e54c0f4e52b617d1e3da1e8925 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:51:24 UTC) from {username='kudu', principal='kudu/127.4.159.190@KRBTEST.COM'} at 127.0.0.1:56520
I20260807 08:51:24.450627 30707 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f0ff9e54c0f4e52b617d1e3da1e8925 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:51:24 UTC) from {username='kudu', principal='kudu/127.4.159.190@KRBTEST.COM'} at 127.0.0.1:48234
I20260807 08:51:24.450829 31301 tablet_replica.cc:330] T 3f0ff9e54c0f4e52b617d1e3da1e8925 P a3bd5d22c741407d94cb6e145688dc38: stopping tablet replica
I20260807 08:51:24.450918 31301 raft_consensus.cc:2229] T 3f0ff9e54c0f4e52b617d1e3da1e8925 P a3bd5d22c741407d94cb6e145688dc38 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:51:24.451006 31301 raft_consensus.cc:2258] T 3f0ff9e54c0f4e52b617d1e3da1e8925 P a3bd5d22c741407d94cb6e145688dc38 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:51:24.451047 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 30605
I20260807 08:51:24.451153 31300 tablet_replica.cc:330] T 3f0ff9e54c0f4e52b617d1e3da1e8925 P c21b7be4cc70482b951e449d84955f05: stopping tablet replica
I20260807 08:51:24.451236 31300 raft_consensus.cc:2229] T 3f0ff9e54c0f4e52b617d1e3da1e8925 P c21b7be4cc70482b951e449d84955f05 [term 1 LEADER]: Raft consensus shutting down.
I20260807 08:51:24.451325 31300 raft_consensus.cc:2258] T 3f0ff9e54c0f4e52b617d1e3da1e8925 P c21b7be4cc70482b951e449d84955f05 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:51:24.451459 31298 tablet_replica.cc:330] T 3f0ff9e54c0f4e52b617d1e3da1e8925 P 90c0cbd4946a4ed9b7308742c5bd12ce: stopping tablet replica
I20260807 08:51:24.451603 31298 raft_consensus.cc:2229] T 3f0ff9e54c0f4e52b617d1e3da1e8925 P 90c0cbd4946a4ed9b7308742c5bd12ce [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:51:24.451702 31298 raft_consensus.cc:2258] T 3f0ff9e54c0f4e52b617d1e3da1e8925 P 90c0cbd4946a4ed9b7308742c5bd12ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:51:24.456884 31298 ts_tablet_manager.cc:1905] T 3f0ff9e54c0f4e52b617d1e3da1e8925 P 90c0cbd4946a4ed9b7308742c5bd12ce: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:51:24.456918 31300 ts_tablet_manager.cc:1905] T 3f0ff9e54c0f4e52b617d1e3da1e8925 P c21b7be4cc70482b951e449d84955f05: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:51:24.458482 31298 ts_tablet_manager.cc:1918] T 3f0ff9e54c0f4e52b617d1e3da1e8925 P 90c0cbd4946a4ed9b7308742c5bd12ce: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:51:24.458559 31298 log.cc:1196] T 3f0ff9e54c0f4e52b617d1e3da1e8925 P 90c0cbd4946a4ed9b7308742c5bd12ce: Deleting WAL directory at /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786092453486433-4734-0/minicluster-data/ts-2/wal/wals/3f0ff9e54c0f4e52b617d1e3da1e8925
I20260807 08:51:24.458832 31298 ts_tablet_manager.cc:1939] T 3f0ff9e54c0f4e52b617d1e3da1e8925 P 90c0cbd4946a4ed9b7308742c5bd12ce: Deleting consensus metadata
I20260807 08:51:24.459177 31300 ts_tablet_manager.cc:1918] T 3f0ff9e54c0f4e52b617d1e3da1e8925 P c21b7be4cc70482b951e449d84955f05: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:51:24.459255 30483 catalog_manager.cc:4841] TS 90c0cbd4946a4ed9b7308742c5bd12ce (127.4.159.131:44427): tablet 3f0ff9e54c0f4e52b617d1e3da1e8925 (table test-table [id=f4b542c1f1fc4d07b47f79c209fc5232]) successfully deleted
I20260807 08:51:24.459321 31300 log.cc:1196] T 3f0ff9e54c0f4e52b617d1e3da1e8925 P c21b7be4cc70482b951e449d84955f05: Deleting WAL directory at /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786092453486433-4734-0/minicluster-data/ts-1/wal/wals/3f0ff9e54c0f4e52b617d1e3da1e8925
I20260807 08:51:24.459734 31300 ts_tablet_manager.cc:1939] T 3f0ff9e54c0f4e52b617d1e3da1e8925 P c21b7be4cc70482b951e449d84955f05: Deleting consensus metadata
I20260807 08:51:24.460213 30485 catalog_manager.cc:4841] TS c21b7be4cc70482b951e449d84955f05 (127.4.159.130:42893): tablet 3f0ff9e54c0f4e52b617d1e3da1e8925 (table test-table [id=f4b542c1f1fc4d07b47f79c209fc5232]) successfully deleted
W20260807 08:51:24.482280 30481 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.159.129:42435 (error 108)
I20260807 08:51:24.482530 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 30816
I20260807 08:51:24.484268 31257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:24.482622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.129:42435 (local address 127.0.0.1:48238)
0807 08:51:24.484167 (+ 1545us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:24.484220 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.129:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1517}
W20260807 08:51:24.484413 30481 catalog_manager.cc:4584] TS a3bd5d22c741407d94cb6e145688dc38 (127.4.159.129:42435): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3f0ff9e54c0f4e52b617d1e3da1e8925: Network error: Client connection negotiation failed: client connection to 127.4.159.129:42435: connect: Connection refused (error 111)
I20260807 08:51:24.488471 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 31021
I20260807 08:51:24.499209 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 30452
2026-08-07T08:51:24Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2033 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-07T08:51:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:51:24Z Disabled control of system clock
I20260807 08:51:24.531451 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786092453486433-4734-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786092453486433-4734-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786092453486433-4734-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.159.190:45265
--webserver_interface=127.4.159.190
--webserver_port=0
--builtin_ntp_servers=127.4.159.148:33773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.159.190:45265
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 08:51:24.611289 31308 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:51:24.611478 31308 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:51:24.611496 31308 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:51:24.612553 31308 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:51:24.612584 31308 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:51:24.612596 31308 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:51:24.612607 31308 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:51:24.612623 31308 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:51:24.613878 31308 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:33773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786092453486433-4734-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786092453486433-4734-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.159.190:45265
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.159.190:45265
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.159.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31308
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786092453486433-4734-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:51:24.614055 31308 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:51:24.614280 31308 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:51:24.618680 31317 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:51:24.618700 31316 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:51:24.618749 31308 server_base.cc:1034] running on GCE node
W20260807 08:51:24.618680 31319 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:51:24.619122 31308 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:51:24.619383 31308 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:51:24.620512 31308 hybrid_clock.cc:648] HybridClock initialized: now 1786092684620501 us; error 28 us; skew 500 ppm
I20260807 08:51:24.621614 31308 webserver.cc:460] Webserver started at http://127.4.159.190:46207/ using document root <none> and password file <none>
I20260807 08:51:24.621838 31308 fs_manager.cc:359] Metadata directory not provided
I20260807 08:51:24.621882 31308 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:51:24.622004 31308 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:51:24.622990 31308 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786092453486433-4734-0/minicluster-data/master-0/data/instance:
uuid: "f72f27ac457c4732a82e53798eb1363c"
format_stamp: "Formatted at 2026-08-07 08:51:24 on dist-test-slave-6zbq"
server_key: "aef98a309a28c7d80226f2756f03d48c"
server_key_iv: "53e6efbd95747e767e4ff7eacd9d68b4"
server_key_version: "encryptionkey@0"
I20260807 08:51:24.623293 31308 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786092453486433-4734-0/minicluster-data/master-0/wal/instance:
uuid: "f72f27ac457c4732a82e53798eb1363c"
format_stamp: "Formatted at 2026-08-07 08:51:24 on dist-test-slave-6zbq"
server_key: "aef98a309a28c7d80226f2756f03d48c"
server_key_iv: "53e6efbd95747e767e4ff7eacd9d68b4"
server_key_version: "encryptionkey@0"
I20260807 08:51:24.624433 31308 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 08:51:24.625064 31329 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:51:24.625183 31308 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 08:51:24.625254 31308 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786092453486433-4734-0/minicluster-data/master-0/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786092453486433-4734-0/minicluster-data/master-0/wal
uuid: "f72f27ac457c4732a82e53798eb1363c"
format_stamp: "Formatted at 2026-08-07 08:51:24 on dist-test-slave-6zbq"
server_key: "aef98a309a28c7d80226f2756f03d48c"
server_key_iv: "53e6efbd95747e767e4ff7eacd9d68b4"
server_key_version: "encryptionkey@0"
I20260807 08:51:24.625319 31308 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786092453486433-4734-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786092453486433-4734-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786092453486433-4734-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:51:24.634864 31308 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:51:24.635084 31308 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:51:24.635195 31308 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:51:24.638711 31308 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.190:45265
I20260807 08:51:24.638752 31412 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.190:45265 every 8 connection(s)
I20260807 08:51:24.639014 31308 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
I20260807 08:51:24.639875 31413 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:51:24.641846 31413 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f72f27ac457c4732a82e53798eb1363c: Bootstrap starting.
I20260807 08:51:24.642373 31413 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f72f27ac457c4732a82e53798eb1363c: Neither blocks nor log segments found. Creating new log.
I20260807 08:51:24.642635 31413 log.cc:824] T 00000000000000000000000000000000 P f72f27ac457c4732a82e53798eb1363c: Log is configured to *not* fsync() on all Append() calls
I20260807 08:51:24.643226 31413 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f72f27ac457c4732a82e53798eb1363c: No bootstrap required, opened a new log
I20260807 08:51:24.644773 31413 raft_consensus.cc:348] T 00000000000000000000000000000000 P f72f27ac457c4732a82e53798eb1363c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f72f27ac457c4732a82e53798eb1363c" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 45265 } }
I20260807 08:51:24.644930 31413 raft_consensus.cc:374] T 00000000000000000000000000000000 P f72f27ac457c4732a82e53798eb1363c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:51:24.644968 31413 raft_consensus.cc:729] T 00000000000000000000000000000000 P f72f27ac457c4732a82e53798eb1363c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f72f27ac457c4732a82e53798eb1363c, State: Initialized, Role: FOLLOWER
I20260807 08:51:24.645072 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 31308
I20260807 08:51:24.645107 31413 consensus_queue.cc:260] T 00000000000000000000000000000000 P f72f27ac457c4732a82e53798eb1363c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f72f27ac457c4732a82e53798eb1363c" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 45265 } }
I20260807 08:51:24.645180 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786092453486433-4734-0/minicluster-data/master-0/wal/instance
I20260807 08:51:24.645182 31413 raft_consensus.cc:388] T 00000000000000000000000000000000 P f72f27ac457c4732a82e53798eb1363c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:51:24.645211 31413 raft_consensus.cc:482] T 00000000000000000000000000000000 P f72f27ac457c4732a82e53798eb1363c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:51:24.645268 31413 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f72f27ac457c4732a82e53798eb1363c [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:51:24.645364 4734 external_mini_cluster.cc:1428] Setting key 84d3a01ab002edf2280cd85f4529fea6
I20260807 08:51:24.645849 31413 raft_consensus.cc:504] T 00000000000000000000000000000000 P f72f27ac457c4732a82e53798eb1363c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f72f27ac457c4732a82e53798eb1363c" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 45265 } }
I20260807 08:51:24.645958 31413 leader_election.cc:302] T 00000000000000000000000000000000 P f72f27ac457c4732a82e53798eb1363c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f72f27ac457c4732a82e53798eb1363c; no voters:
I20260807 08:51:24.646116 31413 leader_election.cc:288] T 00000000000000000000000000000000 P f72f27ac457c4732a82e53798eb1363c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:51:24.646144 31421 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f72f27ac457c4732a82e53798eb1363c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:51:24.646242 31421 raft_consensus.cc:686] T 00000000000000000000000000000000 P f72f27ac457c4732a82e53798eb1363c [term 1 LEADER]: Becoming Leader. State: Replica: f72f27ac457c4732a82e53798eb1363c, State: Running, Role: LEADER
I20260807 08:51:24.646405 31413 sys_catalog.cc:564] T 00000000000000000000000000000000 P f72f27ac457c4732a82e53798eb1363c [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:51:24.646397 31421 consensus_queue.cc:237] T 00000000000000000000000000000000 P f72f27ac457c4732a82e53798eb1363c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f72f27ac457c4732a82e53798eb1363c" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 45265 } }
I20260807 08:51:24.646797 31425 sys_catalog.cc:455] T 00000000000000000000000000000000 P f72f27ac457c4732a82e53798eb1363c [sys.catalog]: SysCatalogTable state changed. Reason: New leader f72f27ac457c4732a82e53798eb1363c. Latest consensus state: current_term: 1 leader_uuid: "f72f27ac457c4732a82e53798eb1363c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f72f27ac457c4732a82e53798eb1363c" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 45265 } } }
I20260807 08:51:24.646883 31423 sys_catalog.cc:455] T 00000000000000000000000000000000 P f72f27ac457c4732a82e53798eb1363c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f72f27ac457c4732a82e53798eb1363c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f72f27ac457c4732a82e53798eb1363c" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 45265 } } }
I20260807 08:51:24.647008 31423 sys_catalog.cc:458] T 00000000000000000000000000000000 P f72f27ac457c4732a82e53798eb1363c [sys.catalog]: This master's current role is: LEADER
I20260807 08:51:24.646960 31425 sys_catalog.cc:458] T 00000000000000000000000000000000 P f72f27ac457c4732a82e53798eb1363c [sys.catalog]: This master's current role is: LEADER
W20260807 08:51:24.648275 31441 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f72f27ac457c4732a82e53798eb1363c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 08:51:24.648332 31441 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 08:51:24.648403 31433 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:51:24.648600 31433 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:51:24.650504 31433 catalog_manager.cc:1347] Generated new cluster ID: 4cb944952e724fd59d4c69117b8b6ef4
I20260807 08:51:24.650557 31433 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:51:24.651000 31420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:24.645822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35322 (local address 127.4.159.190:45265)
0807 08:51:24.646050 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:24.646059 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:24.646089 (+ 30us) server_negotiation.cc:407] Connection header received
0807 08:51:24.646196 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:24.646203 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:24.646274 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:24.646485 (+ 211us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:51:24.646708 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:24.647990 (+ 1282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:24.648964 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:24.649473 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:24.649592 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:24.650274 (+ 682us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:51:24.650278 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:51:24.650278 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:51:24.650521 (+ 243us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:51:24.650564 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:51:24.650580 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:51:24.650641 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:24.650757 (+ 116us) server_negotiation.cc:299] Negotiation successful
0807 08:51:24.650840 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":82,"threads_started":1}
I20260807 08:51:24.673447 31433 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:51:24.674494 31433 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:51:24.679659 31433 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f72f27ac457c4732a82e53798eb1363c: Generated new TSK 0
I20260807 08:51:24.679978 31433 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:51:24.683681 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786092453486433-4734-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786092453486433-4734-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786092453486433-4734-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786092453486433-4734-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.159.129:0
--local_ip_for_outbound_sockets=127.4.159.129
--webserver_interface=127.4.159.129
--webserver_port=0
--tserver_master_addrs=127.4.159.190:45265
--builtin_ntp_servers=127.4.159.148:33773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 08:51:24.785297 31446 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:51:24.785483 31446 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:51:24.785511 31446 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:51:24.786633 31446 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:51:24.786670 31446 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:51:24.786703 31446 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.159.129
I20260807 08:51:24.788009 31446 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:33773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786092453486433-4734-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786092453486433-4734-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.159.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786092453486433-4734-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.159.129
--webserver_port=0
--tserver_master_addrs=127.4.159.190:45265
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31446
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.159.129
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786092453486433-4734-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:51:24.788234 31446 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:51:24.788506 31446 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:51:24.793097 31455 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:51:24.793210 31456 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:51:24.793081 31459 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:51:24.793687 31446 server_base.cc:1034] running on GCE node
I20260807 08:51:24.793867 31446 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:51:24.794075 31446 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:51:24.795210 31446 hybrid_clock.cc:648] HybridClock initialized: now 1786092684795195 us; error 31 us; skew 500 ppm
I20260807 08:51:24.796350 31446 webserver.cc:460] Webserver started at http://127.4.159.129:32933/ using document root <none> and password file <none>
I20260807 08:51:24.796587 31446 fs_manager.cc:359] Metadata directory not provided
I20260807 08:51:24.796630 31446 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:51:24.796753 31446 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:51:24.797727 31446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786092453486433-4734-0/minicluster-data/ts-0/data/instance:
uuid: "1283d518bb82433ca93124123d9e1fd6"
format_stamp: "Formatted at 2026-08-07 08:51:24 on dist-test-slave-6zbq"
server_key: "de6a9574d84162883fc71f84428e0392"
server_key_iv: "006d0af6e281977163a25990354b05a7"
server_key_version: "encryptionkey@0"
I20260807 08:51:24.798064 31446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786092453486433-4734-0/minicluster-data/ts-0/wal/instance:
uuid: "1283d518bb82433ca93124123d9e1fd6"
format_stamp: "Formatted at 2026-08-07 08:51:24 on dist-test-slave-6zbq"
server_key: "de6a9574d84162883fc71f84428e0392"
server_key_iv: "006d0af6e281977163a25990354b05a7"
server_key_version: "encryptionkey@0"
I20260807 08:51:24.799326 31446 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 08:51:24.799911 31469 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:51:24.800067 31446 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 08:51:24.800154 31446 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786092453486433-4734-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786092453486433-4734-0/minicluster-data/ts-0/wal
uuid: "1283d518bb82433ca93124123d9e1fd6"
format_stamp: "Formatted at 2026-08-07 08:51:24 on dist-test-slave-6zbq"
server_key: "de6a9574d84162883fc71f84428e0392"
server_key_iv: "006d0af6e281977163a25990354b05a7"
server_key_version: "encryptionkey@0"
I20260807 08:51:24.800231 31446 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786092453486433-4734-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786092453486433-4734-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786092453486433-4734-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:51:24.821861 31446 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:51:24.822115 31446 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:51:24.822237 31446 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:51:24.822454 31446 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:51:24.822760 31446 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:51:24.822793 31446 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:51:24.822815 31446 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:51:24.822839 31446 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:51:24.828009 31446 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.129:46549
I20260807 08:51:24.828073 31647 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.129:46549 every 8 connection(s)
I20260807 08:51:24.828351 31446 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786092453486433-4734-0/minicluster-data/ts-0/data/info.pb
I20260807 08:51:24.832873 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 31446
I20260807 08:51:24.832978 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786092453486433-4734-0/minicluster-data/ts-0/wal/instance
I20260807 08:51:24.833178 4734 external_mini_cluster.cc:1428] Setting key f440bf5ef26b48a215ed35ae68a429b8
I20260807 08:51:24.833783 31420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:24.829005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.129:45819 (local address 127.4.159.190:45265)
0807 08:51:24.829052 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:24.829054 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:24.829312 (+ 258us) server_negotiation.cc:407] Connection header received
0807 08:51:24.829779 (+ 467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:24.829780 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:24.829791 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:24.829825 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:51:24.830477 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:24.830958 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:24.831837 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:24.832013 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:24.832057 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:24.833485 (+ 1428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:51:24.833487 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:51:24.833487 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:51:24.833492 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:51:24.833508 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:51:24.833513 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:51:24.833561 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:24.833656 (+ 95us) server_negotiation.cc:299] Negotiation successful
0807 08:51:24.833689 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260807 08:51:24.833957 31652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:24.829003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:45265 (local address 127.4.159.129:45819)
0807 08:51:24.829204 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:24.829236 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0807 08:51:24.829682 (+ 446us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:51:24.829881 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:51:24.829884 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:51:24.829891 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:51:24.830410 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:24.830417 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:24.831019 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:24.831019 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:24.831778 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:24.831780 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:24.832075 (+ 295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:24.832076 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:24.832230 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:24.833371 (+ 1141us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 08:51:24.833372 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:51:24.833401 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 08:51:24.833423 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:51:24.833578 (+ 155us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:51:24.833586 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:51:24.833589 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 08:51:24.833650 (+ 61us) client_negotiation.cc:240] Negotiation successful
0807 08:51:24.833751 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":95,"threads_started":1}
I20260807 08:51:24.834089 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786092453486433-4734-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786092453486433-4734-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786092453486433-4734-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786092453486433-4734-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.159.130:0
--local_ip_for_outbound_sockets=127.4.159.130
--webserver_interface=127.4.159.130
--webserver_port=0
--tserver_master_addrs=127.4.159.190:45265
--builtin_ntp_servers=127.4.159.148:33773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 08:51:24.834374 31648 heartbeater.cc:347] Connected to a master server at 127.4.159.190:45265
I20260807 08:51:24.834447 31648 heartbeater.cc:464] Registering TS with master...
I20260807 08:51:24.834578 31648 heartbeater.cc:511] Master 127.4.159.190:45265 requested a full tablet report, sending...
I20260807 08:51:24.835048 31363 ts_manager.cc:194] Registered new tserver with Master: 1283d518bb82433ca93124123d9e1fd6 (127.4.159.129:46549)
I20260807 08:51:24.835810 31363 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.159.129:45819
W20260807 08:51:24.912014 31653 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:51:24.912191 31653 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:51:24.912211 31653 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:51:24.913247 31653 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:51:24.913275 31653 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:51:24.913297 31653 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.159.130
I20260807 08:51:24.914553 31653 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:33773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786092453486433-4734-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786092453486433-4734-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.159.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786092453486433-4734-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.159.130
--webserver_port=0
--tserver_master_addrs=127.4.159.190:45265
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31653
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.159.130
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786092453486433-4734-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:51:24.914719 31653 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:51:24.914949 31653 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:51:24.919310 31666 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:51:24.919373 31653 server_base.cc:1034] running on GCE node
W20260807 08:51:24.919337 31663 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:51:24.919340 31664 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:51:24.919709 31653 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:51:24.919953 31653 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:51:24.921101 31653 hybrid_clock.cc:648] HybridClock initialized: now 1786092684921086 us; error 29 us; skew 500 ppm
I20260807 08:51:24.922281 31653 webserver.cc:460] Webserver started at http://127.4.159.130:33435/ using document root <none> and password file <none>
I20260807 08:51:24.922514 31653 fs_manager.cc:359] Metadata directory not provided
I20260807 08:51:24.922564 31653 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:51:24.922680 31653 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:51:24.923619 31653 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786092453486433-4734-0/minicluster-data/ts-1/data/instance:
uuid: "d9b6d6146fab4d2f9c55f95df4f83ca8"
format_stamp: "Formatted at 2026-08-07 08:51:24 on dist-test-slave-6zbq"
server_key: "afd9ce88fe5b4d7d941eee3943b743cd"
server_key_iv: "2ce9426cfc06c4258eaea1f00bdd80a0"
server_key_version: "encryptionkey@0"
I20260807 08:51:24.923918 31653 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786092453486433-4734-0/minicluster-data/ts-1/wal/instance:
uuid: "d9b6d6146fab4d2f9c55f95df4f83ca8"
format_stamp: "Formatted at 2026-08-07 08:51:24 on dist-test-slave-6zbq"
server_key: "afd9ce88fe5b4d7d941eee3943b743cd"
server_key_iv: "2ce9426cfc06c4258eaea1f00bdd80a0"
server_key_version: "encryptionkey@0"
I20260807 08:51:24.925045 31653 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 08:51:24.925647 31675 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:51:24.925809 31653 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 08:51:24.925905 31653 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786092453486433-4734-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786092453486433-4734-0/minicluster-data/ts-1/wal
uuid: "d9b6d6146fab4d2f9c55f95df4f83ca8"
format_stamp: "Formatted at 2026-08-07 08:51:24 on dist-test-slave-6zbq"
server_key: "afd9ce88fe5b4d7d941eee3943b743cd"
server_key_iv: "2ce9426cfc06c4258eaea1f00bdd80a0"
server_key_version: "encryptionkey@0"
I20260807 08:51:24.925968 31653 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786092453486433-4734-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786092453486433-4734-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786092453486433-4734-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:51:24.936932 31653 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:51:24.937170 31653 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:51:24.937283 31653 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:51:24.937458 31653 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:51:24.937733 31653 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:51:24.937767 31653 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:51:24.937793 31653 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:51:24.937817 31653 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:51:24.943015 31653 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.130:39777
I20260807 08:51:24.943051 31855 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.130:39777 every 8 connection(s)
I20260807 08:51:24.943332 31653 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786092453486433-4734-0/minicluster-data/ts-1/data/info.pb
I20260807 08:51:24.947875 31420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:24.943936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.130:35705 (local address 127.4.159.190:45265)
0807 08:51:24.943995 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:24.943998 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:24.944192 (+ 194us) server_negotiation.cc:407] Connection header received
0807 08:51:24.944605 (+ 413us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:24.944606 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:24.944615 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:24.944663 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:51:24.945264 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:24.945753 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:24.946736 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:24.946925 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:24.946992 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:24.947684 (+ 692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:51:24.947686 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:51:24.947686 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:51:24.947690 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:51:24.947705 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:51:24.947708 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:51:24.947748 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:24.947797 (+ 49us) server_negotiation.cc:299] Negotiation successful
0807 08:51:24.947818 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260807 08:51:24.948027 31858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:24.943922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:45265 (local address 127.4.159.130:35705)
0807 08:51:24.944091 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:24.944121 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0807 08:51:24.944524 (+ 403us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:51:24.944718 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:51:24.944721 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:51:24.944728 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:51:24.945172 (+ 444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:24.945176 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:24.945818 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:24.945818 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:24.946649 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:24.946652 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:24.946994 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:24.946995 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:24.947121 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:24.947565 (+ 444us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 08:51:24.947565 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:51:24.947591 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 08:51:24.947613 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:51:24.947748 (+ 135us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:51:24.947750 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:51:24.947753 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 08:51:24.947793 (+ 40us) client_negotiation.cc:240] Negotiation successful
0807 08:51:24.947863 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":87,"threads_started":1}
I20260807 08:51:24.948562 31856 heartbeater.cc:347] Connected to a master server at 127.4.159.190:45265
I20260807 08:51:24.948640 31856 heartbeater.cc:464] Registering TS with master...
I20260807 08:51:24.948805 31856 heartbeater.cc:511] Master 127.4.159.190:45265 requested a full tablet report, sending...
I20260807 08:51:24.949065 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 31653
I20260807 08:51:24.949172 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786092453486433-4734-0/minicluster-data/ts-1/wal/instance
I20260807 08:51:24.949162 31363 ts_manager.cc:194] Registered new tserver with Master: d9b6d6146fab4d2f9c55f95df4f83ca8 (127.4.159.130:39777)
I20260807 08:51:24.949355 4734 external_mini_cluster.cc:1428] Setting key 85f3e4a2d4716757be34c413699d69e7
I20260807 08:51:24.949604 31363 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.159.130:35705
I20260807 08:51:24.950296 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786092453486433-4734-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786092453486433-4734-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786092453486433-4734-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786092453486433-4734-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.159.131:0
--local_ip_for_outbound_sockets=127.4.159.131
--webserver_interface=127.4.159.131
--webserver_port=0
--tserver_master_addrs=127.4.159.190:45265
--builtin_ntp_servers=127.4.159.148:33773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 08:51:25.029498 31860 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:51:25.029675 31860 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:51:25.029695 31860 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:51:25.030766 31860 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:51:25.030797 31860 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:51:25.030819 31860 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.159.131
I20260807 08:51:25.032055 31860 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:33773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786092453486433-4734-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786092453486433-4734-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.159.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786092453486433-4734-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.159.131
--webserver_port=0
--tserver_master_addrs=127.4.159.190:45265
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31860
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.159.131
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786092453486433-4734-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:51:25.032218 31860 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:51:25.032433 31860 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:51:25.036840 31867 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:51:25.036859 31870 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:51:25.036863 31860 server_base.cc:1034] running on GCE node
W20260807 08:51:25.036847 31868 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:51:25.037264 31860 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:51:25.037472 31860 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:51:25.038604 31860 hybrid_clock.cc:648] HybridClock initialized: now 1786092685038590 us; error 30 us; skew 500 ppm
I20260807 08:51:25.039708 31860 webserver.cc:460] Webserver started at http://127.4.159.131:34521/ using document root <none> and password file <none>
I20260807 08:51:25.039916 31860 fs_manager.cc:359] Metadata directory not provided
I20260807 08:51:25.039958 31860 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:51:25.040081 31860 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:51:25.041026 31860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786092453486433-4734-0/minicluster-data/ts-2/data/instance:
uuid: "d4643b4a0cfd4dcfbcee361011f95f3d"
format_stamp: "Formatted at 2026-08-07 08:51:25 on dist-test-slave-6zbq"
server_key: "6a4e5c549cb30d0afd18fe9339bf3afa"
server_key_iv: "4ce1ef858a0c4e29ce29205ce3e8132f"
server_key_version: "encryptionkey@0"
I20260807 08:51:25.041363 31860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786092453486433-4734-0/minicluster-data/ts-2/wal/instance:
uuid: "d4643b4a0cfd4dcfbcee361011f95f3d"
format_stamp: "Formatted at 2026-08-07 08:51:25 on dist-test-slave-6zbq"
server_key: "6a4e5c549cb30d0afd18fe9339bf3afa"
server_key_iv: "4ce1ef858a0c4e29ce29205ce3e8132f"
server_key_version: "encryptionkey@0"
I20260807 08:51:25.042583 31860 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 08:51:25.043198 31879 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:51:25.043329 31860 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 08:51:25.043406 31860 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786092453486433-4734-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786092453486433-4734-0/minicluster-data/ts-2/wal
uuid: "d4643b4a0cfd4dcfbcee361011f95f3d"
format_stamp: "Formatted at 2026-08-07 08:51:25 on dist-test-slave-6zbq"
server_key: "6a4e5c549cb30d0afd18fe9339bf3afa"
server_key_iv: "4ce1ef858a0c4e29ce29205ce3e8132f"
server_key_version: "encryptionkey@0"
I20260807 08:51:25.043465 31860 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786092453486433-4734-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786092453486433-4734-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786092453486433-4734-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:51:25.052040 31860 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:51:25.052276 31860 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:51:25.052388 31860 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:51:25.052563 31860 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:51:25.052829 31860 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:51:25.052860 31860 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:51:25.052884 31860 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:51:25.052909 31860 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:51:25.057967 31860 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.131:37645
I20260807 08:51:25.058007 32063 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.131:37645 every 8 connection(s)
I20260807 08:51:25.058269 31860 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786092453486433-4734-0/minicluster-data/ts-2/data/info.pb
I20260807 08:51:25.062520 31420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:25.058887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.131:49079 (local address 127.4.159.190:45265)
0807 08:51:25.058951 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:25.058953 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:25.059138 (+ 185us) server_negotiation.cc:407] Connection header received
0807 08:51:25.059543 (+ 405us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:25.059544 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:25.059553 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:25.059599 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:51:25.060151 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:25.060624 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:25.061489 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:25.061656 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:25.061696 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:25.062288 (+ 592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:51:25.062289 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:51:25.062289 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:51:25.062294 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:51:25.062307 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:51:25.062311 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:51:25.062369 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:25.062436 (+ 67us) server_negotiation.cc:299] Negotiation successful
0807 08:51:25.062463 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260807 08:51:25.062666 32071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:25.058865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:45265 (local address 127.4.159.131:49079)
0807 08:51:25.059037 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:25.059065 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0807 08:51:25.059471 (+ 406us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:51:25.059646 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:51:25.059649 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:51:25.059663 (+ 14us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:51:25.060094 (+ 431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:25.060097 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:25.060687 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:25.060687 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:25.061432 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:25.061434 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:25.061708 (+ 274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:25.061709 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:25.061803 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:25.062176 (+ 373us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 08:51:25.062177 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:51:25.062201 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 08:51:25.062222 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:51:25.062368 (+ 146us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:51:25.062371 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:51:25.062374 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 08:51:25.062424 (+ 50us) client_negotiation.cc:240] Negotiation successful
0807 08:51:25.062495 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":84,"threads_started":1}
I20260807 08:51:25.063099 32064 heartbeater.cc:347] Connected to a master server at 127.4.159.190:45265
I20260807 08:51:25.063182 32064 heartbeater.cc:464] Registering TS with master...
I20260807 08:51:25.063313 32064 heartbeater.cc:511] Master 127.4.159.190:45265 requested a full tablet report, sending...
I20260807 08:51:25.063607 31362 ts_manager.cc:194] Registered new tserver with Master: d4643b4a0cfd4dcfbcee361011f95f3d (127.4.159.131:37645)
I20260807 08:51:25.063948 31362 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.159.131:49079
I20260807 08:51:25.064956 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 31860
I20260807 08:51:25.065034 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786092453486433-4734-0/minicluster-data/ts-2/wal/instance
I20260807 08:51:25.065173 4734 external_mini_cluster.cc:1428] Setting key 4064767eb6992720d732d4b9139510d0
I20260807 08:51:25.065804 4734 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 08:51:25.069201 31420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:25.067017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35336 (local address 127.4.159.190:45265)
0807 08:51:25.067068 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:25.067070 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:25.067153 (+ 83us) server_negotiation.cc:407] Connection header received
0807 08:51:25.067188 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:25.067189 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:25.067198 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:25.067240 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:51:25.067352 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:25.067781 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:25.068414 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:25.068574 (+ 160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:25.068613 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:25.069040 (+ 427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:51:25.069041 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:51:25.069041 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:51:25.069045 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:51:25.069056 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:51:25.069059 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:51:25.069092 (+ 33us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:25.069122 (+ 30us) server_negotiation.cc:299] Negotiation successful
0807 08:51:25.069139 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260807 08:51:25.070437 31362 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35336:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 08:51:25.071843 31362 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:51:25.077415 32083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:25.073910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52412 (local address 127.4.159.131:37645)
0807 08:51:25.074050 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:25.074053 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:25.074065 (+ 12us) server_negotiation.cc:407] Connection header received
0807 08:51:25.074144 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:25.074146 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:25.074159 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:25.074242 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:51:25.074403 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:25.075443 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:25.076468 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:25.077072 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:25.077141 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:25.077214 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:25.077315 (+ 101us) server_negotiation.cc:299] Negotiation successful
0807 08:51:25.077346 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":65,"threads_started":1}
I20260807 08:51:25.077415 32078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:25.073728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.131:37645 (local address 127.0.0.1:52412)
0807 08:51:25.073970 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:25.074006 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 08:51:25.074071 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:51:25.074251 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:51:25.074254 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:51:25.074256 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:51:25.074329 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:25.074331 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:25.075538 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:25.075539 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:25.076382 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:25.076385 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:25.077150 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:25.077151 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:25.077226 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:25.077238 (+ 12us) client_negotiation.cc:769] Sending connection context
0807 08:51:25.077303 (+ 65us) client_negotiation.cc:240] Negotiation successful
0807 08:51:25.077348 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230,"mutex_wait_us":41,"thread_start_us":82,"threads_started":1}
I20260807 08:51:25.077919 32081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:25.073734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.129:46549 (local address 127.0.0.1:51538)
0807 08:51:25.074044 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:25.074054 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 08:51:25.074091 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:51:25.074270 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:51:25.074271 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:51:25.074272 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:51:25.074370 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:25.074374 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:25.075600 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:25.075601 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:25.076875 (+ 1274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:25.076877 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:25.077697 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:25.077698 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:25.077754 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:25.077757 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 08:51:25.077803 (+ 46us) client_negotiation.cc:240] Negotiation successful
0807 08:51:25.077831 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"mutex_wait_us":100,"thread_start_us":81,"threads_started":1}
I20260807 08:51:25.077946 32085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:25.073945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51538 (local address 127.4.159.129:46549)
0807 08:51:25.074132 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:25.074136 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:25.074145 (+ 9us) server_negotiation.cc:407] Connection header received
0807 08:51:25.074180 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:25.074183 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:25.074195 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:25.074272 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:51:25.074454 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:25.075534 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:25.076947 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:25.077620 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:25.077689 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:25.077770 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:25.077828 (+ 58us) server_negotiation.cc:299] Negotiation successful
0807 08:51:25.077867 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":80,"threads_started":1}
I20260807 08:51:25.078013 32082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:25.073801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54840 (local address 127.4.159.130:39777)
0807 08:51:25.074450 (+ 649us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:25.074455 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:25.074465 (+ 10us) server_negotiation.cc:407] Connection header received
0807 08:51:25.074505 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:25.074507 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:25.074521 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:25.074587 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:51:25.074751 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:25.075427 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:25.076784 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:25.077506 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:25.077573 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:25.077641 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:25.077895 (+ 254us) server_negotiation.cc:299] Negotiation successful
0807 08:51:25.077936 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":528,"thread_start_us":86,"threads_started":1}
I20260807 08:51:25.078166 32079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:25.073723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.130:39777 (local address 127.0.0.1:54840)
0807 08:51:25.073867 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:25.074171 (+ 304us) client_negotiation.cc:174] Beginning negotiation
0807 08:51:25.074204 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:51:25.074653 (+ 449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:51:25.074654 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:51:25.074656 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:51:25.074694 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:25.074695 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:25.075510 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:25.075511 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:25.076708 (+ 1197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:25.076710 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:25.077673 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:25.077674 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:25.077738 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:25.077742 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 08:51:25.077791 (+ 49us) client_negotiation.cc:240] Negotiation successful
0807 08:51:25.077828 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"mutex_wait_us":27,"thread_start_us":47,"threads_started":1}
I20260807 08:51:25.078331 31958 tablet_service.cc:1467] Processing CreateTablet for tablet af662c42912e469f948a156e4f91edb4 (DEFAULT_TABLE table=test-table [id=245bda248fb24755a9df0240f544ae49]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:51:25.078563 31539 tablet_service.cc:1467] Processing CreateTablet for tablet af662c42912e469f948a156e4f91edb4 (DEFAULT_TABLE table=test-table [id=245bda248fb24755a9df0240f544ae49]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:51:25.078703 31958 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af662c42912e469f948a156e4f91edb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:51:25.078904 31539 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af662c42912e469f948a156e4f91edb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:51:25.078950 31753 tablet_service.cc:1467] Processing CreateTablet for tablet af662c42912e469f948a156e4f91edb4 (DEFAULT_TABLE table=test-table [id=245bda248fb24755a9df0240f544ae49]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:51:25.079269 31753 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af662c42912e469f948a156e4f91edb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:51:25.081199 32086 tablet_bootstrap.cc:492] T af662c42912e469f948a156e4f91edb4 P 1283d518bb82433ca93124123d9e1fd6: Bootstrap starting.
I20260807 08:51:25.081905 32086 tablet_bootstrap.cc:654] T af662c42912e469f948a156e4f91edb4 P 1283d518bb82433ca93124123d9e1fd6: Neither blocks nor log segments found. Creating new log.
I20260807 08:51:25.082120 32086 log.cc:824] T af662c42912e469f948a156e4f91edb4 P 1283d518bb82433ca93124123d9e1fd6: Log is configured to *not* fsync() on all Append() calls
I20260807 08:51:25.082219 32089 tablet_bootstrap.cc:492] T af662c42912e469f948a156e4f91edb4 P d9b6d6146fab4d2f9c55f95df4f83ca8: Bootstrap starting.
I20260807 08:51:25.082607 32087 tablet_bootstrap.cc:492] T af662c42912e469f948a156e4f91edb4 P d4643b4a0cfd4dcfbcee361011f95f3d: Bootstrap starting.
I20260807 08:51:25.082904 32089 tablet_bootstrap.cc:654] T af662c42912e469f948a156e4f91edb4 P d9b6d6146fab4d2f9c55f95df4f83ca8: Neither blocks nor log segments found. Creating new log.
I20260807 08:51:25.083102 32087 tablet_bootstrap.cc:654] T af662c42912e469f948a156e4f91edb4 P d4643b4a0cfd4dcfbcee361011f95f3d: Neither blocks nor log segments found. Creating new log.
I20260807 08:51:25.083175 32089 log.cc:824] T af662c42912e469f948a156e4f91edb4 P d9b6d6146fab4d2f9c55f95df4f83ca8: Log is configured to *not* fsync() on all Append() calls
I20260807 08:51:25.083352 32087 log.cc:824] T af662c42912e469f948a156e4f91edb4 P d4643b4a0cfd4dcfbcee361011f95f3d: Log is configured to *not* fsync() on all Append() calls
I20260807 08:51:25.083819 32089 tablet_bootstrap.cc:492] T af662c42912e469f948a156e4f91edb4 P d9b6d6146fab4d2f9c55f95df4f83ca8: No bootstrap required, opened a new log
I20260807 08:51:25.083861 32086 tablet_bootstrap.cc:492] T af662c42912e469f948a156e4f91edb4 P 1283d518bb82433ca93124123d9e1fd6: No bootstrap required, opened a new log
I20260807 08:51:25.083889 32089 ts_tablet_manager.cc:1397] T af662c42912e469f948a156e4f91edb4 P d9b6d6146fab4d2f9c55f95df4f83ca8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 08:51:25.083916 32086 ts_tablet_manager.cc:1397] T af662c42912e469f948a156e4f91edb4 P 1283d518bb82433ca93124123d9e1fd6: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260807 08:51:25.084021 32087 tablet_bootstrap.cc:492] T af662c42912e469f948a156e4f91edb4 P d4643b4a0cfd4dcfbcee361011f95f3d: No bootstrap required, opened a new log
I20260807 08:51:25.084082 32087 ts_tablet_manager.cc:1397] T af662c42912e469f948a156e4f91edb4 P d4643b4a0cfd4dcfbcee361011f95f3d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 08:51:25.085812 32086 raft_consensus.cc:348] T af662c42912e469f948a156e4f91edb4 P 1283d518bb82433ca93124123d9e1fd6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9b6d6146fab4d2f9c55f95df4f83ca8" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 39777 } } peers { permanent_uuid: "d4643b4a0cfd4dcfbcee361011f95f3d" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 37645 } } peers { permanent_uuid: "1283d518bb82433ca93124123d9e1fd6" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 46549 } }
I20260807 08:51:25.085937 32086 raft_consensus.cc:374] T af662c42912e469f948a156e4f91edb4 P 1283d518bb82433ca93124123d9e1fd6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:51:25.085968 32086 raft_consensus.cc:729] T af662c42912e469f948a156e4f91edb4 P 1283d518bb82433ca93124123d9e1fd6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1283d518bb82433ca93124123d9e1fd6, State: Initialized, Role: FOLLOWER
I20260807 08:51:25.086081 32086 consensus_queue.cc:260] T af662c42912e469f948a156e4f91edb4 P 1283d518bb82433ca93124123d9e1fd6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9b6d6146fab4d2f9c55f95df4f83ca8" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 39777 } } peers { permanent_uuid: "d4643b4a0cfd4dcfbcee361011f95f3d" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 37645 } } peers { permanent_uuid: "1283d518bb82433ca93124123d9e1fd6" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 46549 } }
I20260807 08:51:25.086315 32086 ts_tablet_manager.cc:1428] T af662c42912e469f948a156e4f91edb4 P 1283d518bb82433ca93124123d9e1fd6: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260807 08:51:25.086313 32089 raft_consensus.cc:348] T af662c42912e469f948a156e4f91edb4 P d9b6d6146fab4d2f9c55f95df4f83ca8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9b6d6146fab4d2f9c55f95df4f83ca8" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 39777 } } peers { permanent_uuid: "d4643b4a0cfd4dcfbcee361011f95f3d" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 37645 } } peers { permanent_uuid: "1283d518bb82433ca93124123d9e1fd6" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 46549 } }
I20260807 08:51:25.086459 31648 heartbeater.cc:503] Master 127.4.159.190:45265 was elected leader, sending a full tablet report...
I20260807 08:51:25.086489 32089 raft_consensus.cc:374] T af662c42912e469f948a156e4f91edb4 P d9b6d6146fab4d2f9c55f95df4f83ca8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:51:25.086529 32089 raft_consensus.cc:729] T af662c42912e469f948a156e4f91edb4 P d9b6d6146fab4d2f9c55f95df4f83ca8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9b6d6146fab4d2f9c55f95df4f83ca8, State: Initialized, Role: FOLLOWER
I20260807 08:51:25.086630 32089 consensus_queue.cc:260] T af662c42912e469f948a156e4f91edb4 P d9b6d6146fab4d2f9c55f95df4f83ca8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9b6d6146fab4d2f9c55f95df4f83ca8" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 39777 } } peers { permanent_uuid: "d4643b4a0cfd4dcfbcee361011f95f3d" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 37645 } } peers { permanent_uuid: "1283d518bb82433ca93124123d9e1fd6" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 46549 } }
I20260807 08:51:25.086313 32087 raft_consensus.cc:348] T af662c42912e469f948a156e4f91edb4 P d4643b4a0cfd4dcfbcee361011f95f3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9b6d6146fab4d2f9c55f95df4f83ca8" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 39777 } } peers { permanent_uuid: "d4643b4a0cfd4dcfbcee361011f95f3d" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 37645 } } peers { permanent_uuid: "1283d518bb82433ca93124123d9e1fd6" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 46549 } }
I20260807 08:51:25.086774 32087 raft_consensus.cc:374] T af662c42912e469f948a156e4f91edb4 P d4643b4a0cfd4dcfbcee361011f95f3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:51:25.086823 32087 raft_consensus.cc:729] T af662c42912e469f948a156e4f91edb4 P d4643b4a0cfd4dcfbcee361011f95f3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4643b4a0cfd4dcfbcee361011f95f3d, State: Initialized, Role: FOLLOWER
I20260807 08:51:25.086889 32089 ts_tablet_manager.cc:1428] T af662c42912e469f948a156e4f91edb4 P d9b6d6146fab4d2f9c55f95df4f83ca8: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 08:51:25.086926 32087 consensus_queue.cc:260] T af662c42912e469f948a156e4f91edb4 P d4643b4a0cfd4dcfbcee361011f95f3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9b6d6146fab4d2f9c55f95df4f83ca8" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 39777 } } peers { permanent_uuid: "d4643b4a0cfd4dcfbcee361011f95f3d" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 37645 } } peers { permanent_uuid: "1283d518bb82433ca93124123d9e1fd6" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 46549 } }
I20260807 08:51:25.087013 31856 heartbeater.cc:503] Master 127.4.159.190:45265 was elected leader, sending a full tablet report...
I20260807 08:51:25.087147 32087 ts_tablet_manager.cc:1428] T af662c42912e469f948a156e4f91edb4 P d4643b4a0cfd4dcfbcee361011f95f3d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 08:51:25.087246 32064 heartbeater.cc:503] Master 127.4.159.190:45265 was elected leader, sending a full tablet report...
W20260807 08:51:25.193843 31857 tablet.cc:2369] T af662c42912e469f948a156e4f91edb4 P d9b6d6146fab4d2f9c55f95df4f83ca8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 08:51:25.308785 32067 tablet.cc:2369] T af662c42912e469f948a156e4f91edb4 P d4643b4a0cfd4dcfbcee361011f95f3d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 08:51:25.329013 31650 tablet.cc:2369] T af662c42912e469f948a156e4f91edb4 P 1283d518bb82433ca93124123d9e1fd6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 08:51:25.441001 32094 raft_consensus.cc:482] T af662c42912e469f948a156e4f91edb4 P d9b6d6146fab4d2f9c55f95df4f83ca8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:51:25.441126 32094 raft_consensus.cc:504] T af662c42912e469f948a156e4f91edb4 P d9b6d6146fab4d2f9c55f95df4f83ca8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9b6d6146fab4d2f9c55f95df4f83ca8" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 39777 } } peers { permanent_uuid: "d4643b4a0cfd4dcfbcee361011f95f3d" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 37645 } } peers { permanent_uuid: "1283d518bb82433ca93124123d9e1fd6" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 46549 } }
I20260807 08:51:25.441449 32094 leader_election.cc:288] T af662c42912e469f948a156e4f91edb4 P d9b6d6146fab4d2f9c55f95df4f83ca8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d4643b4a0cfd4dcfbcee361011f95f3d (127.4.159.131:37645), 1283d518bb82433ca93124123d9e1fd6 (127.4.159.129:46549)
I20260807 08:51:25.445158 32083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:25.441724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.130:53949 (local address 127.4.159.131:37645)
0807 08:51:25.441766 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:25.441768 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:25.441775 (+ 7us) server_negotiation.cc:407] Connection header received
0807 08:51:25.441809 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:25.441810 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:25.441817 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:25.441886 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:51:25.442047 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:25.442823 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:25.444267 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:25.444802 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:25.444875 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:25.444883 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:25.445049 (+ 166us) server_negotiation.cc:299] Negotiation successful
0807 08:51:25.445087 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 08:51:25.445169 32085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:25.441715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.130:51325 (local address 127.4.159.129:46549)
0807 08:51:25.441760 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:25.441762 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:25.441853 (+ 91us) server_negotiation.cc:407] Connection header received
0807 08:51:25.441904 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:25.441905 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:25.441917 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:25.441964 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:51:25.442087 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:25.442812 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:25.444260 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:25.444802 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:25.444875 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:25.444883 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:25.445049 (+ 166us) server_negotiation.cc:299] Negotiation successful
0807 08:51:25.445087 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 08:51:25.445161 31858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:25.441640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.131:37645 (local address 127.4.159.130:53949)
0807 08:51:25.441701 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:25.441711 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 08:51:25.441752 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:51:25.441905 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:51:25.441906 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:51:25.441907 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:51:25.441979 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:25.441981 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:25.442886 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:25.442887 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:25.444178 (+ 1291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:25.444182 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:25.444879 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:25.444880 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:25.444975 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:25.444980 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 08:51:25.445027 (+ 47us) client_negotiation.cc:240] Negotiation successful
0807 08:51:25.445071 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42,"mutex_wait_us":43}
I20260807 08:51:25.445168 32104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:25.441641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.129:46549 (local address 127.4.159.130:51325)
0807 08:51:25.441794 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:25.441801 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0807 08:51:25.441849 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:51:25.441975 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:51:25.441975 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:51:25.441977 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:51:25.442028 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:25.442029 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:25.442886 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:25.442887 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:25.444181 (+ 1294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:25.444182 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:25.444879 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:25.444880 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:25.444975 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:25.444980 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 08:51:25.445027 (+ 47us) client_negotiation.cc:240] Negotiation successful
0807 08:51:25.445071 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"mutex_wait_us":63,"thread_start_us":64,"threads_started":1}
I20260807 08:51:25.445537 31575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af662c42912e469f948a156e4f91edb4" candidate_uuid: "d9b6d6146fab4d2f9c55f95df4f83ca8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1283d518bb82433ca93124123d9e1fd6" is_pre_election: true
I20260807 08:51:25.445533 31988 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af662c42912e469f948a156e4f91edb4" candidate_uuid: "d9b6d6146fab4d2f9c55f95df4f83ca8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4643b4a0cfd4dcfbcee361011f95f3d" is_pre_election: true
I20260807 08:51:25.445673 31575 raft_consensus.cc:2454] T af662c42912e469f948a156e4f91edb4 P 1283d518bb82433ca93124123d9e1fd6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9b6d6146fab4d2f9c55f95df4f83ca8 in term 0.
I20260807 08:51:25.445677 31988 raft_consensus.cc:2454] T af662c42912e469f948a156e4f91edb4 P d4643b4a0cfd4dcfbcee361011f95f3d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9b6d6146fab4d2f9c55f95df4f83ca8 in term 0.
I20260807 08:51:25.445894 31680 leader_election.cc:302] T af662c42912e469f948a156e4f91edb4 P d9b6d6146fab4d2f9c55f95df4f83ca8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d4643b4a0cfd4dcfbcee361011f95f3d, d9b6d6146fab4d2f9c55f95df4f83ca8; no voters:
I20260807 08:51:25.446029 32094 raft_consensus.cc:2781] T af662c42912e469f948a156e4f91edb4 P d9b6d6146fab4d2f9c55f95df4f83ca8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:51:25.446098 32094 raft_consensus.cc:482] T af662c42912e469f948a156e4f91edb4 P d9b6d6146fab4d2f9c55f95df4f83ca8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:51:25.446115 32094 raft_consensus.cc:3037] T af662c42912e469f948a156e4f91edb4 P d9b6d6146fab4d2f9c55f95df4f83ca8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:51:25.446902 32094 raft_consensus.cc:504] T af662c42912e469f948a156e4f91edb4 P d9b6d6146fab4d2f9c55f95df4f83ca8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9b6d6146fab4d2f9c55f95df4f83ca8" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 39777 } } peers { permanent_uuid: "d4643b4a0cfd4dcfbcee361011f95f3d" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 37645 } } peers { permanent_uuid: "1283d518bb82433ca93124123d9e1fd6" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 46549 } }
I20260807 08:51:25.447016 32094 leader_election.cc:288] T af662c42912e469f948a156e4f91edb4 P d9b6d6146fab4d2f9c55f95df4f83ca8 [CANDIDATE]: Term 1 election: Requested vote from peers d4643b4a0cfd4dcfbcee361011f95f3d (127.4.159.131:37645), 1283d518bb82433ca93124123d9e1fd6 (127.4.159.129:46549)
I20260807 08:51:25.447155 31575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af662c42912e469f948a156e4f91edb4" candidate_uuid: "d9b6d6146fab4d2f9c55f95df4f83ca8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1283d518bb82433ca93124123d9e1fd6"
I20260807 08:51:25.447171 31988 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af662c42912e469f948a156e4f91edb4" candidate_uuid: "d9b6d6146fab4d2f9c55f95df4f83ca8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4643b4a0cfd4dcfbcee361011f95f3d"
I20260807 08:51:25.447243 31988 raft_consensus.cc:3037] T af662c42912e469f948a156e4f91edb4 P d4643b4a0cfd4dcfbcee361011f95f3d [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:51:25.447240 31575 raft_consensus.cc:3037] T af662c42912e469f948a156e4f91edb4 P 1283d518bb82433ca93124123d9e1fd6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:51:25.447933 31988 raft_consensus.cc:2454] T af662c42912e469f948a156e4f91edb4 P d4643b4a0cfd4dcfbcee361011f95f3d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9b6d6146fab4d2f9c55f95df4f83ca8 in term 1.
I20260807 08:51:25.448017 31575 raft_consensus.cc:2454] T af662c42912e469f948a156e4f91edb4 P 1283d518bb82433ca93124123d9e1fd6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9b6d6146fab4d2f9c55f95df4f83ca8 in term 1.
I20260807 08:51:25.448091 31680 leader_election.cc:302] T af662c42912e469f948a156e4f91edb4 P d9b6d6146fab4d2f9c55f95df4f83ca8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d4643b4a0cfd4dcfbcee361011f95f3d, d9b6d6146fab4d2f9c55f95df4f83ca8; no voters:
I20260807 08:51:25.448171 32094 raft_consensus.cc:2781] T af662c42912e469f948a156e4f91edb4 P d9b6d6146fab4d2f9c55f95df4f83ca8 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:51:25.448318 32094 raft_consensus.cc:686] T af662c42912e469f948a156e4f91edb4 P d9b6d6146fab4d2f9c55f95df4f83ca8 [term 1 LEADER]: Becoming Leader. State: Replica: d9b6d6146fab4d2f9c55f95df4f83ca8, State: Running, Role: LEADER
I20260807 08:51:25.448408 32094 consensus_queue.cc:237] T af662c42912e469f948a156e4f91edb4 P d9b6d6146fab4d2f9c55f95df4f83ca8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9b6d6146fab4d2f9c55f95df4f83ca8" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 39777 } } peers { permanent_uuid: "d4643b4a0cfd4dcfbcee361011f95f3d" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 37645 } } peers { permanent_uuid: "1283d518bb82433ca93124123d9e1fd6" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 46549 } }
I20260807 08:51:25.449187 31363 catalog_manager.cc:5495] T af662c42912e469f948a156e4f91edb4 P d9b6d6146fab4d2f9c55f95df4f83ca8 reported cstate change: term changed from 0 to 1, leader changed from <none> to d9b6d6146fab4d2f9c55f95df4f83ca8 (127.4.159.130). New cstate: current_term: 1 leader_uuid: "d9b6d6146fab4d2f9c55f95df4f83ca8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9b6d6146fab4d2f9c55f95df4f83ca8" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 39777 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d4643b4a0cfd4dcfbcee361011f95f3d" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 37645 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1283d518bb82433ca93124123d9e1fd6" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 46549 } health_report { overall_health: UNKNOWN } } }
I20260807 08:51:25.514181 32082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:25.512253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54852 (local address 127.4.159.130:39777)
0807 08:51:25.512302 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:25.512303 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:25.512310 (+ 7us) server_negotiation.cc:407] Connection header received
0807 08:51:25.512344 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:25.512345 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:25.512361 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:25.512418 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:51:25.512509 (+ 91us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:25.512966 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:25.513628 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:25.513804 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:25.513848 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:25.513942 (+ 94us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:51:25.514030 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:51:25.514083 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:25.514099 (+ 16us) server_negotiation.cc:299] Negotiation successful
0807 08:51:25.514112 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260807 08:51:25.515761 31988 raft_consensus.cc:1261] T af662c42912e469f948a156e4f91edb4 P d4643b4a0cfd4dcfbcee361011f95f3d [term 1 FOLLOWER]: Refusing update from remote peer d9b6d6146fab4d2f9c55f95df4f83ca8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:51:25.515797 31575 raft_consensus.cc:1261] T af662c42912e469f948a156e4f91edb4 P 1283d518bb82433ca93124123d9e1fd6 [term 1 FOLLOWER]: Refusing update from remote peer d9b6d6146fab4d2f9c55f95df4f83ca8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:51:25.516094 32105 consensus_queue.cc:1035] T af662c42912e469f948a156e4f91edb4 P d9b6d6146fab4d2f9c55f95df4f83ca8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4643b4a0cfd4dcfbcee361011f95f3d" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 37645 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:51:25.516222 32094 consensus_queue.cc:1035] T af662c42912e469f948a156e4f91edb4 P d9b6d6146fab4d2f9c55f95df4f83ca8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1283d518bb82433ca93124123d9e1fd6" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 46549 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:51:25.518332 32108 mvcc.cc:205] Tried to move back new op lower bound from 7315835639870431232 to 7315835639597252608. Current Snapshot: MvccSnapshot[applied={T|T < 7315835639870431232}]
I20260807 08:51:25.518790 32111 mvcc.cc:205] Tried to move back new op lower bound from 7315835639870431232 to 7315835639597252608. Current Snapshot: MvccSnapshot[applied={T|T < 7315835639870431232 or (T in {7315835639870431232})}]
I20260807 08:51:25.519541 32109 mvcc.cc:205] Tried to move back new op lower bound from 7315835639870431232 to 7315835639597252608. Current Snapshot: MvccSnapshot[applied={T|T < 7315835639870431232 or (T in {7315835639870431232})}]
I20260807 08:51:25.520404 31362 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35336:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:51:25.520496 31362 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35336:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:51:25.521184 31362 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f72f27ac457c4732a82e53798eb1363c: Sending DeleteTablet for 3 replicas of tablet af662c42912e469f948a156e4f91edb4
I20260807 08:51:25.521466 31753 tablet_service.cc:1514] Processing DeleteTablet for tablet af662c42912e469f948a156e4f91edb4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:51:25 UTC) from {username='slave'} at 127.0.0.1:54840
I20260807 08:51:25.521507 31539 tablet_service.cc:1514] Processing DeleteTablet for tablet af662c42912e469f948a156e4f91edb4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:51:25 UTC) from {username='slave'} at 127.0.0.1:51538
I20260807 08:51:25.521734 32121 tablet_replica.cc:330] T af662c42912e469f948a156e4f91edb4 P 1283d518bb82433ca93124123d9e1fd6: stopping tablet replica
I20260807 08:51:25.521844 31958 tablet_service.cc:1514] Processing DeleteTablet for tablet af662c42912e469f948a156e4f91edb4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:51:25 UTC) from {username='slave'} at 127.0.0.1:52412
I20260807 08:51:25.521873 32121 raft_consensus.cc:2229] T af662c42912e469f948a156e4f91edb4 P 1283d518bb82433ca93124123d9e1fd6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:51:25.521893 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 31446
I20260807 08:51:25.528069 32120 tablet_replica.cc:330] T af662c42912e469f948a156e4f91edb4 P d9b6d6146fab4d2f9c55f95df4f83ca8: stopping tablet replica
I20260807 08:51:25.528163 32120 raft_consensus.cc:2229] T af662c42912e469f948a156e4f91edb4 P d9b6d6146fab4d2f9c55f95df4f83ca8 [term 1 LEADER]: Raft consensus shutting down.
I20260807 08:51:25.528255 32120 raft_consensus.cc:2258] T af662c42912e469f948a156e4f91edb4 P d9b6d6146fab4d2f9c55f95df4f83ca8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:51:25.528626 32120 ts_tablet_manager.cc:1905] T af662c42912e469f948a156e4f91edb4 P d9b6d6146fab4d2f9c55f95df4f83ca8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:51:25.529971 32120 ts_tablet_manager.cc:1918] T af662c42912e469f948a156e4f91edb4 P d9b6d6146fab4d2f9c55f95df4f83ca8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:51:25.530033 32120 log.cc:1196] T af662c42912e469f948a156e4f91edb4 P d9b6d6146fab4d2f9c55f95df4f83ca8: Deleting WAL directory at /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786092453486433-4734-0/minicluster-data/ts-1/wal/wals/af662c42912e469f948a156e4f91edb4
I20260807 08:51:25.530249 32120 ts_tablet_manager.cc:1939] T af662c42912e469f948a156e4f91edb4 P d9b6d6146fab4d2f9c55f95df4f83ca8: Deleting consensus metadata
I20260807 08:51:25.530695 31330 catalog_manager.cc:4841] TS d9b6d6146fab4d2f9c55f95df4f83ca8 (127.4.159.130:39777): tablet af662c42912e469f948a156e4f91edb4 (table test-table [id=245bda248fb24755a9df0240f544ae49]) successfully deleted
W20260807 08:51:25.534758 31332 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.159.129:46549 (error 108)
I20260807 08:51:25.534852 32122 tablet_replica.cc:330] T af662c42912e469f948a156e4f91edb4 P d4643b4a0cfd4dcfbcee361011f95f3d: stopping tablet replica
I20260807 08:51:25.534955 32122 raft_consensus.cc:2229] T af662c42912e469f948a156e4f91edb4 P d4643b4a0cfd4dcfbcee361011f95f3d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:51:25.535058 32122 raft_consensus.cc:2258] T af662c42912e469f948a156e4f91edb4 P d4643b4a0cfd4dcfbcee361011f95f3d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:51:25.535265 32078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:25.535150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.129:46549 (local address 127.0.0.1:51542)
0807 08:51:25.535202 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:25.535238 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.129:46549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260807 08:51:25.535360 31332 catalog_manager.cc:4584] TS 1283d518bb82433ca93124123d9e1fd6 (127.4.159.129:46549): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet af662c42912e469f948a156e4f91edb4: Network error: Client connection negotiation failed: client connection to 127.4.159.129:46549: connect: Connection refused (error 111)
I20260807 08:51:25.535440 32122 ts_tablet_manager.cc:1905] T af662c42912e469f948a156e4f91edb4 P d4643b4a0cfd4dcfbcee361011f95f3d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:51:25.535663 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 31653
I20260807 08:51:25.536748 32122 ts_tablet_manager.cc:1918] T af662c42912e469f948a156e4f91edb4 P d4643b4a0cfd4dcfbcee361011f95f3d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:51:25.536809 32122 log.cc:1196] T af662c42912e469f948a156e4f91edb4 P d4643b4a0cfd4dcfbcee361011f95f3d: Deleting WAL directory at /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786092453486433-4734-0/minicluster-data/ts-2/wal/wals/af662c42912e469f948a156e4f91edb4
I20260807 08:51:25.537000 32122 ts_tablet_manager.cc:1939] T af662c42912e469f948a156e4f91edb4 P d4643b4a0cfd4dcfbcee361011f95f3d: Deleting consensus metadata
I20260807 08:51:25.537870 31333 catalog_manager.cc:4841] TS d4643b4a0cfd4dcfbcee361011f95f3d (127.4.159.131:37645): tablet af662c42912e469f948a156e4f91edb4 (table test-table [id=245bda248fb24755a9df0240f544ae49]) successfully deleted
I20260807 08:51:25.543532 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 31860
I20260807 08:51:25.550794 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 31308
2026-08-07T08:51:25Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1045 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786092453486433-4734-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:51:25 dist-test-slave-6zbq krb5kdc[32126](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:51:25 dist-test-slave-6zbq krb5kdc[32126](info): set up 2 sockets
Aug 07 08:51:25 dist-test-slave-6zbq krb5kdc[32126](info): commencing operation
krb5kdc: starting...
W20260807 08:51:26.675191 4734 mini_kdc.cc:121] Time spent starting KDC: real 1.104s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786092453486433-4734-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786092453486433-4734-0/krb5kdc/test-user.keytab.
Aug 07 08:51:26 dist-test-slave-6zbq krb5kdc[32126](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092686, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:51:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:51:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.159.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.159.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.159.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab.
Entry for principal kudu/127.4.159.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab.
I20260807 08:51:26.779155 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786092453486433-4734-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786092453486433-4734-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786092453486433-4734-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.159.190:40709
--webserver_interface=127.4.159.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab
--principal=kudu/127.4.159.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:38595
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.159.190:40709
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:51:26.857475 32146 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:51:26.857667 32146 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:51:26.857702 32146 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:51:26.858851 32146 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:51:26.858889 32146 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:51:26.858908 32146 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:51:26.858924 32146 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:51:26.858939 32146 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:51:26.860375 32146 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:38595
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786092453486433-4734-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786092453486433-4734-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.159.190:40709
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab
--principal=kudu/127.4.159.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.159.190:40709
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.159.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32146
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786092453486433-4734-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:51:26.860600 32146 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:51:26.860802 32146 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:51:26.865186 32153 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:51:26.865185 32156 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:51:26.865375 32146 server_base.cc:1034] running on GCE node
W20260807 08:51:26.865231 32160 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:51:26.865666 32146 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:51:26.865943 32146 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:51:26.867076 32146 hybrid_clock.cc:648] HybridClock initialized: now 1786092686867064 us; error 27 us; skew 500 ppm
Aug 07 08:51:26 dist-test-slave-6zbq krb5kdc[32126](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092686, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:51:26.869233 32146 init.cc:377] Logged in from keytab as kudu/127.4.159.190@KRBTEST.COM (short username kudu)
I20260807 08:51:26.869611 32146 webserver.cc:460] Webserver started at http://127.4.159.190:46715/ using document root <none> and password file <none>
I20260807 08:51:26.869837 32146 fs_manager.cc:359] Metadata directory not provided
I20260807 08:51:26.869899 32146 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:51:26.870018 32146 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:51:26.871001 32146 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786092453486433-4734-0/minicluster-data/master-0/data/instance:
uuid: "c645265f895c4952bd7911cdc193a00b"
format_stamp: "Formatted at 2026-08-07 08:51:26 on dist-test-slave-6zbq"
server_key: "32d811c35adcb50b372a7859c2d0447d"
server_key_iv: "29a6c13d2e06b6a81ef9d043e043e3d7"
server_key_version: "encryptionkey@0"
I20260807 08:51:26.871304 32146 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786092453486433-4734-0/minicluster-data/master-0/wal/instance:
uuid: "c645265f895c4952bd7911cdc193a00b"
format_stamp: "Formatted at 2026-08-07 08:51:26 on dist-test-slave-6zbq"
server_key: "32d811c35adcb50b372a7859c2d0447d"
server_key_iv: "29a6c13d2e06b6a81ef9d043e043e3d7"
server_key_version: "encryptionkey@0"
I20260807 08:51:26.872478 32146 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 08:51:26.873119 32169 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:51:26.873265 32146 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 08:51:26.873345 32146 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786092453486433-4734-0/minicluster-data/master-0/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786092453486433-4734-0/minicluster-data/master-0/wal
uuid: "c645265f895c4952bd7911cdc193a00b"
format_stamp: "Formatted at 2026-08-07 08:51:26 on dist-test-slave-6zbq"
server_key: "32d811c35adcb50b372a7859c2d0447d"
server_key_iv: "29a6c13d2e06b6a81ef9d043e043e3d7"
server_key_version: "encryptionkey@0"
I20260807 08:51:26.873401 32146 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786092453486433-4734-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786092453486433-4734-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786092453486433-4734-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:51:26.891563 32146 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:51:26.893061 32146 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:51:26.893184 32146 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:51:26.896773 32146 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.190:40709
I20260807 08:51:26.896812 32241 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.190:40709 every 8 connection(s)
I20260807 08:51:26.897091 32146 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
I20260807 08:51:26.897842 32242 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:51:26.899875 32242 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c645265f895c4952bd7911cdc193a00b: Bootstrap starting.
I20260807 08:51:26.900363 32242 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c645265f895c4952bd7911cdc193a00b: Neither blocks nor log segments found. Creating new log.
I20260807 08:51:26.900580 32242 log.cc:824] T 00000000000000000000000000000000 P c645265f895c4952bd7911cdc193a00b: Log is configured to *not* fsync() on all Append() calls
I20260807 08:51:26.901180 32242 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c645265f895c4952bd7911cdc193a00b: No bootstrap required, opened a new log
I20260807 08:51:26.902750 32242 raft_consensus.cc:348] T 00000000000000000000000000000000 P c645265f895c4952bd7911cdc193a00b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c645265f895c4952bd7911cdc193a00b" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 40709 } }
I20260807 08:51:26.902874 32242 raft_consensus.cc:374] T 00000000000000000000000000000000 P c645265f895c4952bd7911cdc193a00b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:51:26.902895 32242 raft_consensus.cc:729] T 00000000000000000000000000000000 P c645265f895c4952bd7911cdc193a00b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c645265f895c4952bd7911cdc193a00b, State: Initialized, Role: FOLLOWER
I20260807 08:51:26.902989 32242 consensus_queue.cc:260] T 00000000000000000000000000000000 P c645265f895c4952bd7911cdc193a00b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c645265f895c4952bd7911cdc193a00b" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 40709 } }
I20260807 08:51:26.903035 32242 raft_consensus.cc:388] T 00000000000000000000000000000000 P c645265f895c4952bd7911cdc193a00b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:51:26.903057 32242 raft_consensus.cc:482] T 00000000000000000000000000000000 P c645265f895c4952bd7911cdc193a00b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:51:26.903086 32242 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c645265f895c4952bd7911cdc193a00b [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:51:26.903295 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 32146
I20260807 08:51:26.903409 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786092453486433-4734-0/minicluster-data/master-0/wal/instance
I20260807 08:51:26.903602 4734 external_mini_cluster.cc:1428] Setting key 18f23be970f69f211d005273e8fa6e57
I20260807 08:51:26.903697 32242 raft_consensus.cc:504] T 00000000000000000000000000000000 P c645265f895c4952bd7911cdc193a00b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c645265f895c4952bd7911cdc193a00b" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 40709 } }
I20260807 08:51:26.903797 32242 leader_election.cc:302] T 00000000000000000000000000000000 P c645265f895c4952bd7911cdc193a00b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c645265f895c4952bd7911cdc193a00b; no voters:
I20260807 08:51:26.903936 32242 leader_election.cc:288] T 00000000000000000000000000000000 P c645265f895c4952bd7911cdc193a00b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:51:26.903972 32247 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c645265f895c4952bd7911cdc193a00b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:51:26.904170 32247 raft_consensus.cc:686] T 00000000000000000000000000000000 P c645265f895c4952bd7911cdc193a00b [term 1 LEADER]: Becoming Leader. State: Replica: c645265f895c4952bd7911cdc193a00b, State: Running, Role: LEADER
I20260807 08:51:26.904234 32242 sys_catalog.cc:564] T 00000000000000000000000000000000 P c645265f895c4952bd7911cdc193a00b [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:51:26.904266 32247 consensus_queue.cc:237] T 00000000000000000000000000000000 P c645265f895c4952bd7911cdc193a00b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c645265f895c4952bd7911cdc193a00b" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 40709 } }
I20260807 08:51:26.904603 32254 sys_catalog.cc:455] T 00000000000000000000000000000000 P c645265f895c4952bd7911cdc193a00b [sys.catalog]: SysCatalogTable state changed. Reason: New leader c645265f895c4952bd7911cdc193a00b. Latest consensus state: current_term: 1 leader_uuid: "c645265f895c4952bd7911cdc193a00b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c645265f895c4952bd7911cdc193a00b" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 40709 } } }
I20260807 08:51:26.904680 32253 sys_catalog.cc:455] T 00000000000000000000000000000000 P c645265f895c4952bd7911cdc193a00b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c645265f895c4952bd7911cdc193a00b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c645265f895c4952bd7911cdc193a00b" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 40709 } } }
I20260807 08:51:26.904765 32253 sys_catalog.cc:458] T 00000000000000000000000000000000 P c645265f895c4952bd7911cdc193a00b [sys.catalog]: This master's current role is: LEADER
I20260807 08:51:26.904696 32254 sys_catalog.cc:458] T 00000000000000000000000000000000 P c645265f895c4952bd7911cdc193a00b [sys.catalog]: This master's current role is: LEADER
W20260807 08:51:26.906025 32271 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c645265f895c4952bd7911cdc193a00b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 08:51:26.906068 32271 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 08:51:26.906116 32264 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:51:26.906283 32264 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:51:26.909301 32264 catalog_manager.cc:1347] Generated new cluster ID: a07199c2fe244c3590bcd8e03e7c2e09
I20260807 08:51:26.909354 32264 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 08:51:26 dist-test-slave-6zbq krb5kdc[32126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092686, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.159.190@KRBTEST.COM
I20260807 08:51:26.914464 32252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:26.904001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45618 (local address 127.4.159.190:40709)
0807 08:51:26.904166 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:26.904175 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:26.904523 (+ 348us) server_negotiation.cc:407] Connection header received
0807 08:51:26.904643 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:26.904650 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:26.904704 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:26.904847 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:51:26.905583 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:26.906925 (+ 1342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:26.908004 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:26.908583 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:26.908686 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:26.911187 (+ 2501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:51:26.911191 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:51:26.911192 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:51:26.911203 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:51:26.912954 (+ 1751us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:51:26.913341 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:51:26.913344 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:51:26.913344 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:51:26.913446 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:51:26.913632 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:51:26.913633 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:51:26.913633 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:51:26.913885 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:51:26.914016 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:26.914130 (+ 114us) server_negotiation.cc:299] Negotiation successful
0807 08:51:26.914259 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":81,"threads_started":1}
I20260807 08:51:26.922284 32264 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:51:26.922744 32264 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:51:26.930202 32264 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c645265f895c4952bd7911cdc193a00b: Generated new TSK 0
I20260807 08:51:26.930356 32264 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.159.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.159.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.159.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.129.keytab.
Entry for principal kudu/127.4.159.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.129.keytab.
I20260807 08:51:26.956785 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786092453486433-4734-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786092453486433-4734-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786092453486433-4734-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786092453486433-4734-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.159.129:0
--local_ip_for_outbound_sockets=127.4.159.129
--webserver_interface=127.4.159.129
--webserver_port=0
--tserver_master_addrs=127.4.159.190:40709
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.129.keytab
--principal=kudu/127.4.159.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:38595
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:51:27.035307 32282 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:51:27.035495 32282 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:51:27.035516 32282 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:51:27.036618 32282 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:51:27.036657 32282 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:51:27.036681 32282 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.159.129
I20260807 08:51:27.038064 32282 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:38595
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786092453486433-4734-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786092453486433-4734-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.129.keytab
--principal=kudu/127.4.159.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.159.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786092453486433-4734-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.159.129
--webserver_port=0
--tserver_master_addrs=127.4.159.190:40709
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32282
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.159.129
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786092453486433-4734-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:51:27.038270 32282 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:51:27.038487 32282 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:51:27.042991 32293 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:51:27.042966 32291 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:51:27.042976 32290 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:51:27.043213 32282 server_base.cc:1034] running on GCE node
I20260807 08:51:27.043398 32282 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:51:27.043612 32282 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:51:27.044783 32282 hybrid_clock.cc:648] HybridClock initialized: now 1786092687044765 us; error 32 us; skew 500 ppm
Aug 07 08:51:27 dist-test-slave-6zbq krb5kdc[32126](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092687, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:51:27.047071 32282 init.cc:377] Logged in from keytab as kudu/127.4.159.129@KRBTEST.COM (short username kudu)
I20260807 08:51:27.047503 32282 webserver.cc:460] Webserver started at http://127.4.159.129:34923/ using document root <none> and password file <none>
I20260807 08:51:27.047689 32282 fs_manager.cc:359] Metadata directory not provided
I20260807 08:51:27.047735 32282 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:51:27.047856 32282 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:51:27.048808 32282 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786092453486433-4734-0/minicluster-data/ts-0/data/instance:
uuid: "b2603ff4776e4e348f4bfbd67f20014e"
format_stamp: "Formatted at 2026-08-07 08:51:27 on dist-test-slave-6zbq"
server_key: "f63d6c1df554b73f62a3d36a34bfea17"
server_key_iv: "2a8efd37d089556d62f512b3b4700469"
server_key_version: "encryptionkey@0"
I20260807 08:51:27.049134 32282 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786092453486433-4734-0/minicluster-data/ts-0/wal/instance:
uuid: "b2603ff4776e4e348f4bfbd67f20014e"
format_stamp: "Formatted at 2026-08-07 08:51:27 on dist-test-slave-6zbq"
server_key: "f63d6c1df554b73f62a3d36a34bfea17"
server_key_iv: "2a8efd37d089556d62f512b3b4700469"
server_key_version: "encryptionkey@0"
I20260807 08:51:27.050324 32282 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 08:51:27.051013 32303 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:51:27.051172 32282 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 08:51:27.051254 32282 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786092453486433-4734-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786092453486433-4734-0/minicluster-data/ts-0/wal
uuid: "b2603ff4776e4e348f4bfbd67f20014e"
format_stamp: "Formatted at 2026-08-07 08:51:27 on dist-test-slave-6zbq"
server_key: "f63d6c1df554b73f62a3d36a34bfea17"
server_key_iv: "2a8efd37d089556d62f512b3b4700469"
server_key_version: "encryptionkey@0"
I20260807 08:51:27.051314 32282 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786092453486433-4734-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786092453486433-4734-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786092453486433-4734-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:51:27.067019 32282 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:51:27.068588 32282 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:51:27.068717 32282 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:51:27.068902 32282 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:51:27.069172 32282 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:51:27.069205 32282 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:51:27.069231 32282 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:51:27.069255 32282 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:51:27.074738 32282 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.129:44271
I20260807 08:51:27.074798 32486 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.129:44271 every 8 connection(s)
I20260807 08:51:27.075052 32282 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786092453486433-4734-0/minicluster-data/ts-0/data/info.pb
Aug 07 08:51:27 dist-test-slave-6zbq krb5kdc[32126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092687, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.129@KRBTEST.COM for kudu/127.4.159.190@KRBTEST.COM
I20260807 08:51:27.081194 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 32282
I20260807 08:51:27.081306 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786092453486433-4734-0/minicluster-data/ts-0/wal/instance
I20260807 08:51:27.081518 4734 external_mini_cluster.cc:1428] Setting key dc174637df7e9d154889f9401e95c03d
I20260807 08:51:27.083447 32252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:27.075655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.129:53571 (local address 127.4.159.190:40709)
0807 08:51:27.075698 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:27.075700 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:27.075877 (+ 177us) server_negotiation.cc:407] Connection header received
0807 08:51:27.076336 (+ 459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:27.076337 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:27.076351 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:27.076407 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:51:27.077262 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:27.077767 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:27.078757 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:27.078949 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:27.078999 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:27.080808 (+ 1809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:51:27.080810 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:51:27.080810 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:51:27.080816 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:51:27.082230 (+ 1414us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:51:27.082607 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:51:27.082608 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:51:27.082609 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:51:27.082679 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:51:27.082878 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:51:27.082878 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:51:27.082879 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:51:27.083014 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:51:27.083070 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:27.083270 (+ 200us) server_negotiation.cc:299] Negotiation successful
0807 08:51:27.083360 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 08:51:27.083494 32489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:27.075603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:40709 (local address 127.4.159.129:53571)
0807 08:51:27.075771 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:27.075811 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 08:51:27.076242 (+ 431us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:51:27.076469 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:51:27.076472 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:51:27.077005 (+ 533us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:51:27.077184 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:27.077188 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:27.077859 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:27.077860 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:27.078687 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:27.078690 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:27.079007 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:27.079008 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:27.079097 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:27.079460 (+ 363us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:51:27.079468 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:51:27.080708 (+ 1240us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:51:27.082302 (+ 1594us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:51:27.082308 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:51:27.082310 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:51:27.082542 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:51:27.082735 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:51:27.082736 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:51:27.082744 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:51:27.082829 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:51:27.083068 (+ 239us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:51:27.083070 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:51:27.083166 (+ 96us) client_negotiation.cc:769] Sending connection context
0807 08:51:27.083235 (+ 69us) client_negotiation.cc:240] Negotiation successful
0807 08:51:27.083348 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":85,"threads_started":1}
I20260807 08:51:27.084205 32487 heartbeater.cc:347] Connected to a master server at 127.4.159.190:40709
I20260807 08:51:27.084285 32487 heartbeater.cc:464] Registering TS with master...
I20260807 08:51:27.084438 32487 heartbeater.cc:511] Master 127.4.159.190:40709 requested a full tablet report, sending...
I20260807 08:51:27.084908 32195 ts_manager.cc:194] Registered new tserver with Master: b2603ff4776e4e348f4bfbd67f20014e (127.4.159.129:44271)
I20260807 08:51:27.085704 32195 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.159.129@KRBTEST.COM'} at 127.4.159.129:53571
WARNING: no policy specified for kudu/127.4.159.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.159.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.159.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.130.keytab.
Entry for principal kudu/127.4.159.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.130.keytab.
I20260807 08:51:27.104229 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786092453486433-4734-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786092453486433-4734-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786092453486433-4734-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786092453486433-4734-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.159.130:0
--local_ip_for_outbound_sockets=127.4.159.130
--webserver_interface=127.4.159.130
--webserver_port=0
--tserver_master_addrs=127.4.159.190:40709
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.130.keytab
--principal=kudu/127.4.159.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:38595
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:51:27.182955 32492 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:51:27.183135 32492 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:51:27.183156 32492 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:51:27.184244 32492 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:51:27.184283 32492 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:51:27.184310 32492 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.159.130
I20260807 08:51:27.185936 32492 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:38595
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786092453486433-4734-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786092453486433-4734-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.130.keytab
--principal=kudu/127.4.159.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.159.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786092453486433-4734-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.159.130
--webserver_port=0
--tserver_master_addrs=127.4.159.190:40709
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32492
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.159.130
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786092453486433-4734-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:51:27.186178 32492 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:51:27.186417 32492 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:51:27.190944 32505 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:51:27.191042 32506 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:51:27.190971 32509 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:51:27.191028 32492 server_base.cc:1034] running on GCE node
I20260807 08:51:27.191322 32492 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:51:27.191550 32492 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:51:27.192714 32492 hybrid_clock.cc:648] HybridClock initialized: now 1786092687192697 us; error 27 us; skew 500 ppm
Aug 07 08:51:27 dist-test-slave-6zbq krb5kdc[32126](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092687, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:51:27.194964 32492 init.cc:377] Logged in from keytab as kudu/127.4.159.130@KRBTEST.COM (short username kudu)
I20260807 08:51:27.195340 32492 webserver.cc:460] Webserver started at http://127.4.159.130:43201/ using document root <none> and password file <none>
I20260807 08:51:27.195538 32492 fs_manager.cc:359] Metadata directory not provided
I20260807 08:51:27.195585 32492 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:51:27.195708 32492 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:51:27.196669 32492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786092453486433-4734-0/minicluster-data/ts-1/data/instance:
uuid: "de72cfe5d9384ef6906ffdb72de75060"
format_stamp: "Formatted at 2026-08-07 08:51:27 on dist-test-slave-6zbq"
server_key: "d8cc850a22b7a5630903888d71ff3360"
server_key_iv: "1ac6cb432c7086e99f75a60db944d1f4"
server_key_version: "encryptionkey@0"
I20260807 08:51:27.196975 32492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786092453486433-4734-0/minicluster-data/ts-1/wal/instance:
uuid: "de72cfe5d9384ef6906ffdb72de75060"
format_stamp: "Formatted at 2026-08-07 08:51:27 on dist-test-slave-6zbq"
server_key: "d8cc850a22b7a5630903888d71ff3360"
server_key_iv: "1ac6cb432c7086e99f75a60db944d1f4"
server_key_version: "encryptionkey@0"
I20260807 08:51:27.198154 32492 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 08:51:27.198827 32516 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:51:27.198967 32492 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 08:51:27.199045 32492 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786092453486433-4734-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786092453486433-4734-0/minicluster-data/ts-1/wal
uuid: "de72cfe5d9384ef6906ffdb72de75060"
format_stamp: "Formatted at 2026-08-07 08:51:27 on dist-test-slave-6zbq"
server_key: "d8cc850a22b7a5630903888d71ff3360"
server_key_iv: "1ac6cb432c7086e99f75a60db944d1f4"
server_key_version: "encryptionkey@0"
I20260807 08:51:27.199103 32492 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786092453486433-4734-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786092453486433-4734-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786092453486433-4734-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:51:27.211794 32492 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:51:27.213301 32492 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:51:27.213426 32492 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:51:27.213610 32492 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:51:27.213877 32492 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:51:27.213909 32492 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:51:27.213934 32492 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:51:27.213960 32492 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:51:27.219164 32492 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.130:39555
I20260807 08:51:27.219200 32692 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.130:39555 every 8 connection(s)
I20260807 08:51:27.219458 32492 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786092453486433-4734-0/minicluster-data/ts-1/data/info.pb
Aug 07 08:51:27 dist-test-slave-6zbq krb5kdc[32126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092687, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.130@KRBTEST.COM for kudu/127.4.159.190@KRBTEST.COM
I20260807 08:51:27.227258 32252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:27.220049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.130:46335 (local address 127.4.159.190:40709)
0807 08:51:27.220090 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:27.220092 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:27.220291 (+ 199us) server_negotiation.cc:407] Connection header received
0807 08:51:27.220729 (+ 438us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:27.220730 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:27.220743 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:27.220796 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:51:27.221678 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:27.222159 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:27.223128 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:27.223326 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:27.223381 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:27.225212 (+ 1831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:51:27.225213 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:51:27.225213 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:51:27.225220 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:51:27.226233 (+ 1013us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:51:27.226589 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:51:27.226590 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:51:27.226590 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:51:27.226638 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:51:27.226814 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:51:27.226814 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:51:27.226814 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:51:27.226905 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:51:27.226949 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:27.227130 (+ 181us) server_negotiation.cc:299] Negotiation successful
0807 08:51:27.227195 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 08:51:27.227387 32696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:27.220029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:40709 (local address 127.4.159.130:46335)
0807 08:51:27.220188 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:27.220219 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0807 08:51:27.220641 (+ 422us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:51:27.220865 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:51:27.220869 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:51:27.221412 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:51:27.221581 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:27.221586 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:27.222240 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:27.222241 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:27.223048 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:27.223051 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:27.223394 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:27.223395 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:27.223523 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:27.223886 (+ 363us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:51:27.223893 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:51:27.225131 (+ 1238us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:51:27.226294 (+ 1163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:51:27.226300 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:51:27.226301 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:51:27.226530 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:51:27.226682 (+ 152us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:51:27.226682 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:51:27.226682 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:51:27.226774 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:51:27.226945 (+ 171us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:51:27.226947 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:51:27.227044 (+ 97us) client_negotiation.cc:769] Sending connection context
0807 08:51:27.227115 (+ 71us) client_negotiation.cc:240] Negotiation successful
0807 08:51:27.227220 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":79,"threads_started":1}
I20260807 08:51:27.227811 32693 heartbeater.cc:347] Connected to a master server at 127.4.159.190:40709
I20260807 08:51:27.227888 32693 heartbeater.cc:464] Registering TS with master...
I20260807 08:51:27.228039 32693 heartbeater.cc:511] Master 127.4.159.190:40709 requested a full tablet report, sending...
I20260807 08:51:27.228371 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 32492
I20260807 08:51:27.228399 32195 ts_manager.cc:194] Registered new tserver with Master: de72cfe5d9384ef6906ffdb72de75060 (127.4.159.130:39555)
I20260807 08:51:27.228482 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786092453486433-4734-0/minicluster-data/ts-1/wal/instance
I20260807 08:51:27.228665 4734 external_mini_cluster.cc:1428] Setting key f2e6af20089d8f492329a2a75bd5194a
I20260807 08:51:27.228848 32195 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.159.130@KRBTEST.COM'} at 127.4.159.130:46335
WARNING: no policy specified for kudu/127.4.159.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.159.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.159.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.131.keytab.
Entry for principal kudu/127.4.159.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.131.keytab.
I20260807 08:51:27.254295 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786092453486433-4734-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786092453486433-4734-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786092453486433-4734-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786092453486433-4734-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.159.131:0
--local_ip_for_outbound_sockets=127.4.159.131
--webserver_interface=127.4.159.131
--webserver_port=0
--tserver_master_addrs=127.4.159.190:40709
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.131.keytab
--principal=kudu/127.4.159.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:38595
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:51:27.335306 32700 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:51:27.335492 32700 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:51:27.335523 32700 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:51:27.336609 32700 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:51:27.336647 32700 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:51:27.336681 32700 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.159.131
I20260807 08:51:27.338085 32700 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:38595
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786092453486433-4734-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786092453486433-4734-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.131.keytab
--principal=kudu/127.4.159.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.159.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786092453486433-4734-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.159.131
--webserver_port=0
--tserver_master_addrs=127.4.159.190:40709
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32700
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.159.131
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786092453486433-4734-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:51:27.338307 32700 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:51:27.338523 32700 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:51:27.343113 32708 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:51:27.343117 32710 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:51:27.343124 32713 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:51:27.343175 32700 server_base.cc:1034] running on GCE node
I20260807 08:51:27.343498 32700 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:51:27.343688 32700 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:51:27.344818 32700 hybrid_clock.cc:648] HybridClock initialized: now 1786092687344809 us; error 26 us; skew 500 ppm
Aug 07 08:51:27 dist-test-slave-6zbq krb5kdc[32126](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092687, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:51:27.347007 32700 init.cc:377] Logged in from keytab as kudu/127.4.159.131@KRBTEST.COM (short username kudu)
I20260807 08:51:27.347416 32700 webserver.cc:460] Webserver started at http://127.4.159.131:33469/ using document root <none> and password file <none>
I20260807 08:51:27.347620 32700 fs_manager.cc:359] Metadata directory not provided
I20260807 08:51:27.347666 32700 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:51:27.347790 32700 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:51:27.348742 32700 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786092453486433-4734-0/minicluster-data/ts-2/data/instance:
uuid: "034fa62af1634ee4b1877650bf94b9b2"
format_stamp: "Formatted at 2026-08-07 08:51:27 on dist-test-slave-6zbq"
server_key: "1b2fc567ea9b87179ebcee0aa9ff8eca"
server_key_iv: "d433ae5ac5f2319ddb38db92a186d47d"
server_key_version: "encryptionkey@0"
I20260807 08:51:27.349048 32700 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786092453486433-4734-0/minicluster-data/ts-2/wal/instance:
uuid: "034fa62af1634ee4b1877650bf94b9b2"
format_stamp: "Formatted at 2026-08-07 08:51:27 on dist-test-slave-6zbq"
server_key: "1b2fc567ea9b87179ebcee0aa9ff8eca"
server_key_iv: "d433ae5ac5f2319ddb38db92a186d47d"
server_key_version: "encryptionkey@0"
I20260807 08:51:27.350190 32700 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 08:51:27.350884 32722 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:51:27.351025 32700 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 08:51:27.351104 32700 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786092453486433-4734-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786092453486433-4734-0/minicluster-data/ts-2/wal
uuid: "034fa62af1634ee4b1877650bf94b9b2"
format_stamp: "Formatted at 2026-08-07 08:51:27 on dist-test-slave-6zbq"
server_key: "1b2fc567ea9b87179ebcee0aa9ff8eca"
server_key_iv: "d433ae5ac5f2319ddb38db92a186d47d"
server_key_version: "encryptionkey@0"
I20260807 08:51:27.351167 32700 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786092453486433-4734-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786092453486433-4734-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786092453486433-4734-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:51:27.360093 32700 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:51:27.361595 32700 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:51:27.361712 32700 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:51:27.361891 32700 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:51:27.362138 32700 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:51:27.362170 32700 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:51:27.362195 32700 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:51:27.362219 32700 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:51:27.367727 32700 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.131:38637
I20260807 08:51:27.367761 424 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.131:38637 every 8 connection(s)
I20260807 08:51:27.368029 32700 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786092453486433-4734-0/minicluster-data/ts-2/data/info.pb
I20260807 08:51:27.369647 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 32700
I20260807 08:51:27.369740 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786092453486433-4734-0/minicluster-data/ts-2/wal/instance
I20260807 08:51:27.369925 4734 external_mini_cluster.cc:1428] Setting key 3105ef4dc0b1ad3db496c42083d5a4e0
Aug 07 08:51:27 dist-test-slave-6zbq krb5kdc[32126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092687, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.131@KRBTEST.COM for kudu/127.4.159.190@KRBTEST.COM
I20260807 08:51:27.376293 32252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:27.368618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.131:47629 (local address 127.4.159.190:40709)
0807 08:51:27.368670 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:27.368672 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:27.368828 (+ 156us) server_negotiation.cc:407] Connection header received
0807 08:51:27.369235 (+ 407us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:27.369236 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:27.369245 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:27.369297 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:51:27.370375 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:27.371015 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:27.371939 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:27.372135 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:27.372196 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:27.374001 (+ 1805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:51:27.374002 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:51:27.374003 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:51:27.374010 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:51:27.375089 (+ 1079us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:51:27.375426 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:51:27.375428 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:51:27.375428 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:51:27.375490 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:51:27.375705 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:51:27.375706 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:51:27.375706 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:51:27.375822 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:51:27.375877 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:27.376114 (+ 237us) server_negotiation.cc:299] Negotiation successful
0807 08:51:27.376203 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260807 08:51:27.376410 431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:27.368576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:40709 (local address 127.4.159.131:47629)
0807 08:51:27.368730 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:27.368767 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 08:51:27.369160 (+ 393us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:51:27.369354 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:51:27.369357 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:51:27.370060 (+ 703us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:51:27.370269 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:27.370276 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:27.371094 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:27.371095 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:27.371858 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:27.371861 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:27.372203 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:27.372204 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:27.372331 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:27.372727 (+ 396us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:51:27.372734 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:51:27.373916 (+ 1182us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:51:27.375159 (+ 1243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:51:27.375166 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:51:27.375168 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:51:27.375362 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:51:27.375548 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:51:27.375549 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:51:27.375549 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:51:27.375642 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:51:27.375877 (+ 235us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:51:27.375883 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:51:27.375999 (+ 116us) client_negotiation.cc:769] Sending connection context
0807 08:51:27.376084 (+ 85us) client_negotiation.cc:240] Negotiation successful
0807 08:51:27.376223 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":72,"threads_started":1}
I20260807 08:51:27.376848 426 heartbeater.cc:347] Connected to a master server at 127.4.159.190:40709
I20260807 08:51:27.376926 426 heartbeater.cc:464] Registering TS with master...
I20260807 08:51:27.377075 426 heartbeater.cc:511] Master 127.4.159.190:40709 requested a full tablet report, sending...
I20260807 08:51:27.377435 32192 ts_manager.cc:194] Registered new tserver with Master: 034fa62af1634ee4b1877650bf94b9b2 (127.4.159.131:38637)
I20260807 08:51:27.377776 32192 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.159.131@KRBTEST.COM'} at 127.4.159.131:47629
I20260807 08:51:27.381512 4734 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 08:51:27.387775 32252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:27.382760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45626 (local address 127.4.159.190:40709)
0807 08:51:27.382806 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:27.382808 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:27.382904 (+ 96us) server_negotiation.cc:407] Connection header received
0807 08:51:27.382960 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:27.382960 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:27.382969 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:27.383002 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:51:27.383500 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:27.383931 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:27.384583 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:27.384749 (+ 166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:27.384788 (+ 39us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:27.385895 (+ 1107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:51:27.385896 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:51:27.385896 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:51:27.385901 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:51:27.386905 (+ 1004us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:51:27.387219 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:51:27.387219 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:51:27.387220 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:51:27.387263 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:51:27.387398 (+ 135us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:51:27.387399 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:51:27.387399 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:51:27.387498 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:51:27.387570 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:27.387660 (+ 90us) server_negotiation.cc:299] Negotiation successful
0807 08:51:27.387705 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260807 08:51:27.389155 32192 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45626:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 08:51:27.390604 32192 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:51:27.396005 442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:27.392436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.131:38637 (local address 127.0.0.1:44626)
0807 08:51:27.392779 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:27.392788 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 08:51:27.392835 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:51:27.392998 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:51:27.392999 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:51:27.393003 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:51:27.393077 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:27.393078 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:27.394036 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:27.394038 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:27.394966 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:27.394969 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:27.395725 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:27.395726 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:27.395810 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:27.395823 (+ 13us) client_negotiation.cc:769] Sending connection context
0807 08:51:27.395889 (+ 66us) client_negotiation.cc:240] Negotiation successful
0807 08:51:27.395933 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":326,"mutex_wait_us":2,"thread_start_us":62,"threads_started":1}
I20260807 08:51:27.396013 446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:27.392628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44626 (local address 127.4.159.131:38637)
0807 08:51:27.392784 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:27.392788 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:27.392830 (+ 42us) server_negotiation.cc:407] Connection header received
0807 08:51:27.392902 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:27.392904 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:27.392916 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:27.392991 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:51:27.393141 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:27.393932 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:27.395048 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:27.395648 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:27.395717 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:27.395768 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:27.395905 (+ 137us) server_negotiation.cc:299] Negotiation successful
0807 08:51:27.395940 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":66,"threads_started":1}
I20260807 08:51:27.396539 439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:27.392435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.129:44271 (local address 127.0.0.1:37832)
0807 08:51:27.393081 (+ 646us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:27.393091 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 08:51:27.393182 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:51:27.393399 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:51:27.393400 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:51:27.393402 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:51:27.393443 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:27.393446 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:27.394436 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:27.394438 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:27.395511 (+ 1073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:27.395514 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:27.396333 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:27.396334 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:27.396394 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:27.396398 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 08:51:27.396441 (+ 43us) client_negotiation.cc:240] Negotiation successful
0807 08:51:27.396478 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":633,"mutex_wait_us":20,"thread_start_us":69,"threads_started":1}
I20260807 08:51:27.396584 444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:27.392535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37832 (local address 127.4.159.129:44271)
0807 08:51:27.392733 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:27.392737 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:27.393151 (+ 414us) server_negotiation.cc:407] Connection header received
0807 08:51:27.393295 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:27.393298 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:27.393316 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:27.393393 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:51:27.393503 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:27.394295 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:27.395592 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:27.396257 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:27.396321 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:27.396374 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:27.396465 (+ 91us) server_negotiation.cc:299] Negotiation successful
0807 08:51:27.396509 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":75,"threads_started":1}
I20260807 08:51:27.396984 318 tablet_service.cc:1467] Processing CreateTablet for tablet 2c85626eb2574e8a80cd48c3d031fa51 (DEFAULT_TABLE table=test-table [id=6b727daa71034b7faae9e4fce3886d4d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:51:27.397275 440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:27.392434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.130:39555 (local address 127.0.0.1:37554)
0807 08:51:27.392647 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:27.392683 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 08:51:27.392744 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:51:27.392996 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:51:27.392999 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:51:27.393001 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:51:27.393074 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:27.393076 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:27.395105 (+ 2029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:27.395106 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:27.396069 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:27.396072 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:27.397050 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:27.397051 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:27.397112 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:27.397116 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 08:51:27.397178 (+ 62us) client_negotiation.cc:240] Negotiation successful
0807 08:51:27.397212 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"mutex_wait_us":2,"thread_start_us":58,"threads_started":1}
I20260807 08:51:27.397305 318 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c85626eb2574e8a80cd48c3d031fa51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:51:27.397500 447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:27.392672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37554 (local address 127.4.159.130:39555)
0807 08:51:27.392844 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:27.392848 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:27.392856 (+ 8us) server_negotiation.cc:407] Connection header received
0807 08:51:27.392892 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:27.392894 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:27.392910 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:27.392989 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:51:27.394042 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:27.394985 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:27.396160 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:27.396836 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:27.396906 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:27.396957 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:27.397382 (+ 425us) server_negotiation.cc:299] Negotiation successful
0807 08:51:27.397425 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":77,"threads_started":1}
I20260807 08:51:27.397429 32383 tablet_service.cc:1467] Processing CreateTablet for tablet 2c85626eb2574e8a80cd48c3d031fa51 (DEFAULT_TABLE table=test-table [id=6b727daa71034b7faae9e4fce3886d4d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:51:27.397739 32383 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c85626eb2574e8a80cd48c3d031fa51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:51:27.399085 32590 tablet_service.cc:1467] Processing CreateTablet for tablet 2c85626eb2574e8a80cd48c3d031fa51 (DEFAULT_TABLE table=test-table [id=6b727daa71034b7faae9e4fce3886d4d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:51:27.399412 32590 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c85626eb2574e8a80cd48c3d031fa51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:51:27.400249 448 tablet_bootstrap.cc:492] T 2c85626eb2574e8a80cd48c3d031fa51 P 034fa62af1634ee4b1877650bf94b9b2: Bootstrap starting.
I20260807 08:51:27.400650 450 tablet_bootstrap.cc:492] T 2c85626eb2574e8a80cd48c3d031fa51 P b2603ff4776e4e348f4bfbd67f20014e: Bootstrap starting.
I20260807 08:51:27.400940 448 tablet_bootstrap.cc:654] T 2c85626eb2574e8a80cd48c3d031fa51 P 034fa62af1634ee4b1877650bf94b9b2: Neither blocks nor log segments found. Creating new log.
I20260807 08:51:27.401355 450 tablet_bootstrap.cc:654] T 2c85626eb2574e8a80cd48c3d031fa51 P b2603ff4776e4e348f4bfbd67f20014e: Neither blocks nor log segments found. Creating new log.
I20260807 08:51:27.401540 448 log.cc:824] T 2c85626eb2574e8a80cd48c3d031fa51 P 034fa62af1634ee4b1877650bf94b9b2: Log is configured to *not* fsync() on all Append() calls
I20260807 08:51:27.401933 450 log.cc:824] T 2c85626eb2574e8a80cd48c3d031fa51 P b2603ff4776e4e348f4bfbd67f20014e: Log is configured to *not* fsync() on all Append() calls
I20260807 08:51:27.402297 454 tablet_bootstrap.cc:492] T 2c85626eb2574e8a80cd48c3d031fa51 P de72cfe5d9384ef6906ffdb72de75060: Bootstrap starting.
I20260807 08:51:27.402670 450 tablet_bootstrap.cc:492] T 2c85626eb2574e8a80cd48c3d031fa51 P b2603ff4776e4e348f4bfbd67f20014e: No bootstrap required, opened a new log
I20260807 08:51:27.402734 450 ts_tablet_manager.cc:1397] T 2c85626eb2574e8a80cd48c3d031fa51 P b2603ff4776e4e348f4bfbd67f20014e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 08:51:27.402956 454 tablet_bootstrap.cc:654] T 2c85626eb2574e8a80cd48c3d031fa51 P de72cfe5d9384ef6906ffdb72de75060: Neither blocks nor log segments found. Creating new log.
I20260807 08:51:27.403488 454 log.cc:824] T 2c85626eb2574e8a80cd48c3d031fa51 P de72cfe5d9384ef6906ffdb72de75060: Log is configured to *not* fsync() on all Append() calls
I20260807 08:51:27.404157 454 tablet_bootstrap.cc:492] T 2c85626eb2574e8a80cd48c3d031fa51 P de72cfe5d9384ef6906ffdb72de75060: No bootstrap required, opened a new log
I20260807 08:51:27.404275 454 ts_tablet_manager.cc:1397] T 2c85626eb2574e8a80cd48c3d031fa51 P de72cfe5d9384ef6906ffdb72de75060: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 08:51:27.404587 450 raft_consensus.cc:348] T 2c85626eb2574e8a80cd48c3d031fa51 P b2603ff4776e4e348f4bfbd67f20014e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2603ff4776e4e348f4bfbd67f20014e" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 44271 } } peers { permanent_uuid: "034fa62af1634ee4b1877650bf94b9b2" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 38637 } } peers { permanent_uuid: "de72cfe5d9384ef6906ffdb72de75060" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 39555 } }
I20260807 08:51:27.404775 450 raft_consensus.cc:374] T 2c85626eb2574e8a80cd48c3d031fa51 P b2603ff4776e4e348f4bfbd67f20014e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:51:27.404819 450 raft_consensus.cc:729] T 2c85626eb2574e8a80cd48c3d031fa51 P b2603ff4776e4e348f4bfbd67f20014e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2603ff4776e4e348f4bfbd67f20014e, State: Initialized, Role: FOLLOWER
I20260807 08:51:27.404928 450 consensus_queue.cc:260] T 2c85626eb2574e8a80cd48c3d031fa51 P b2603ff4776e4e348f4bfbd67f20014e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2603ff4776e4e348f4bfbd67f20014e" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 44271 } } peers { permanent_uuid: "034fa62af1634ee4b1877650bf94b9b2" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 38637 } } peers { permanent_uuid: "de72cfe5d9384ef6906ffdb72de75060" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 39555 } }
I20260807 08:51:27.405216 450 ts_tablet_manager.cc:1428] T 2c85626eb2574e8a80cd48c3d031fa51 P b2603ff4776e4e348f4bfbd67f20014e: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260807 08:51:27.405239 32487 heartbeater.cc:503] Master 127.4.159.190:40709 was elected leader, sending a full tablet report...
I20260807 08:51:27.405616 448 tablet_bootstrap.cc:492] T 2c85626eb2574e8a80cd48c3d031fa51 P 034fa62af1634ee4b1877650bf94b9b2: No bootstrap required, opened a new log
I20260807 08:51:27.405681 448 ts_tablet_manager.cc:1397] T 2c85626eb2574e8a80cd48c3d031fa51 P 034fa62af1634ee4b1877650bf94b9b2: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260807 08:51:27.406319 454 raft_consensus.cc:348] T 2c85626eb2574e8a80cd48c3d031fa51 P de72cfe5d9384ef6906ffdb72de75060 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2603ff4776e4e348f4bfbd67f20014e" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 44271 } } peers { permanent_uuid: "034fa62af1634ee4b1877650bf94b9b2" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 38637 } } peers { permanent_uuid: "de72cfe5d9384ef6906ffdb72de75060" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 39555 } }
I20260807 08:51:27.406518 454 raft_consensus.cc:374] T 2c85626eb2574e8a80cd48c3d031fa51 P de72cfe5d9384ef6906ffdb72de75060 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:51:27.406562 454 raft_consensus.cc:729] T 2c85626eb2574e8a80cd48c3d031fa51 P de72cfe5d9384ef6906ffdb72de75060 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de72cfe5d9384ef6906ffdb72de75060, State: Initialized, Role: FOLLOWER
I20260807 08:51:27.406695 454 consensus_queue.cc:260] T 2c85626eb2574e8a80cd48c3d031fa51 P de72cfe5d9384ef6906ffdb72de75060 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2603ff4776e4e348f4bfbd67f20014e" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 44271 } } peers { permanent_uuid: "034fa62af1634ee4b1877650bf94b9b2" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 38637 } } peers { permanent_uuid: "de72cfe5d9384ef6906ffdb72de75060" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 39555 } }
I20260807 08:51:27.406953 454 ts_tablet_manager.cc:1428] T 2c85626eb2574e8a80cd48c3d031fa51 P de72cfe5d9384ef6906ffdb72de75060: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260807 08:51:27.406985 32693 heartbeater.cc:503] Master 127.4.159.190:40709 was elected leader, sending a full tablet report...
I20260807 08:51:27.407490 448 raft_consensus.cc:348] T 2c85626eb2574e8a80cd48c3d031fa51 P 034fa62af1634ee4b1877650bf94b9b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2603ff4776e4e348f4bfbd67f20014e" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 44271 } } peers { permanent_uuid: "034fa62af1634ee4b1877650bf94b9b2" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 38637 } } peers { permanent_uuid: "de72cfe5d9384ef6906ffdb72de75060" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 39555 } }
I20260807 08:51:27.407611 448 raft_consensus.cc:374] T 2c85626eb2574e8a80cd48c3d031fa51 P 034fa62af1634ee4b1877650bf94b9b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:51:27.407647 448 raft_consensus.cc:729] T 2c85626eb2574e8a80cd48c3d031fa51 P 034fa62af1634ee4b1877650bf94b9b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 034fa62af1634ee4b1877650bf94b9b2, State: Initialized, Role: FOLLOWER
I20260807 08:51:27.407753 448 consensus_queue.cc:260] T 2c85626eb2574e8a80cd48c3d031fa51 P 034fa62af1634ee4b1877650bf94b9b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2603ff4776e4e348f4bfbd67f20014e" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 44271 } } peers { permanent_uuid: "034fa62af1634ee4b1877650bf94b9b2" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 38637 } } peers { permanent_uuid: "de72cfe5d9384ef6906ffdb72de75060" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 39555 } }
I20260807 08:51:27.408007 448 ts_tablet_manager.cc:1428] T 2c85626eb2574e8a80cd48c3d031fa51 P 034fa62af1634ee4b1877650bf94b9b2: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260807 08:51:27.408025 426 heartbeater.cc:503] Master 127.4.159.190:40709 was elected leader, sending a full tablet report...
W20260807 08:51:27.469959 32695 tablet.cc:2369] T 2c85626eb2574e8a80cd48c3d031fa51 P de72cfe5d9384ef6906ffdb72de75060: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 08:51:27.566125 461 raft_consensus.cc:482] T 2c85626eb2574e8a80cd48c3d031fa51 P de72cfe5d9384ef6906ffdb72de75060 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:51:27.566257 461 raft_consensus.cc:504] T 2c85626eb2574e8a80cd48c3d031fa51 P de72cfe5d9384ef6906ffdb72de75060 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2603ff4776e4e348f4bfbd67f20014e" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 44271 } } peers { permanent_uuid: "034fa62af1634ee4b1877650bf94b9b2" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 38637 } } peers { permanent_uuid: "de72cfe5d9384ef6906ffdb72de75060" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 39555 } }
I20260807 08:51:27.566676 461 leader_election.cc:288] T 2c85626eb2574e8a80cd48c3d031fa51 P de72cfe5d9384ef6906ffdb72de75060 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b2603ff4776e4e348f4bfbd67f20014e (127.4.159.129:44271), 034fa62af1634ee4b1877650bf94b9b2 (127.4.159.131:38637)
I20260807 08:51:27.569403 444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:27.566918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.130:48821 (local address 127.4.159.129:44271)
0807 08:51:27.566977 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:27.566979 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:27.566988 (+ 9us) server_negotiation.cc:407] Connection header received
0807 08:51:27.567022 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:27.567022 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:27.567032 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:27.567118 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:51:27.567274 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:27.567751 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:27.568732 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:27.569092 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:27.569154 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:27.569164 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:27.569303 (+ 139us) server_negotiation.cc:299] Negotiation successful
0807 08:51:27.569339 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260807 08:51:27.569435 32696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:27.566845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.129:44271 (local address 127.4.159.130:48821)
0807 08:51:27.566892 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:27.566904 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 08:51:27.566951 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:51:27.567130 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:51:27.567131 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:51:27.567133 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:51:27.567206 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:27.567208 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:27.567825 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:27.567826 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:27.568649 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:27.568651 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:27.569165 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:27.569166 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:27.569244 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:27.569246 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 08:51:27.569290 (+ 44us) client_negotiation.cc:240] Negotiation successful
0807 08:51:27.569330 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260807 08:51:27.569445 446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:27.566959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.130:41727 (local address 127.4.159.131:38637)
0807 08:51:27.567009 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:27.567010 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:27.567112 (+ 102us) server_negotiation.cc:407] Connection header received
0807 08:51:27.567157 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:27.567158 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:27.567168 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:27.567215 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:51:27.567356 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:27.567810 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:27.568734 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:27.569112 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:27.569174 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:27.569182 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:27.569347 (+ 165us) server_negotiation.cc:299] Negotiation successful
0807 08:51:27.569386 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260807 08:51:27.569449 463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:27.566853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.131:38637 (local address 127.4.159.130:41727)
0807 08:51:27.567045 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:27.567053 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0807 08:51:27.567097 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:51:27.567222 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:51:27.567223 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:51:27.567224 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:51:27.567296 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:27.567297 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:27.567876 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:27.567877 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:27.568650 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:27.568653 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:27.569180 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:27.569181 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:27.569258 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:27.569261 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 08:51:27.569305 (+ 44us) client_negotiation.cc:240] Negotiation successful
0807 08:51:27.569344 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"mutex_wait_us":75,"thread_start_us":96,"threads_started":1}
I20260807 08:51:27.569754 355 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c85626eb2574e8a80cd48c3d031fa51" candidate_uuid: "de72cfe5d9384ef6906ffdb72de75060" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "034fa62af1634ee4b1877650bf94b9b2" is_pre_election: true
I20260807 08:51:27.569783 32419 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c85626eb2574e8a80cd48c3d031fa51" candidate_uuid: "de72cfe5d9384ef6906ffdb72de75060" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2603ff4776e4e348f4bfbd67f20014e" is_pre_election: true
I20260807 08:51:27.569904 355 raft_consensus.cc:2454] T 2c85626eb2574e8a80cd48c3d031fa51 P 034fa62af1634ee4b1877650bf94b9b2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de72cfe5d9384ef6906ffdb72de75060 in term 0.
I20260807 08:51:27.569922 32419 raft_consensus.cc:2454] T 2c85626eb2574e8a80cd48c3d031fa51 P b2603ff4776e4e348f4bfbd67f20014e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de72cfe5d9384ef6906ffdb72de75060 in term 0.
I20260807 08:51:27.570104 32523 leader_election.cc:302] T 2c85626eb2574e8a80cd48c3d031fa51 P de72cfe5d9384ef6906ffdb72de75060 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 034fa62af1634ee4b1877650bf94b9b2, de72cfe5d9384ef6906ffdb72de75060; no voters:
I20260807 08:51:27.570257 461 raft_consensus.cc:2781] T 2c85626eb2574e8a80cd48c3d031fa51 P de72cfe5d9384ef6906ffdb72de75060 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:51:27.570298 461 raft_consensus.cc:482] T 2c85626eb2574e8a80cd48c3d031fa51 P de72cfe5d9384ef6906ffdb72de75060 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:51:27.570319 461 raft_consensus.cc:3037] T 2c85626eb2574e8a80cd48c3d031fa51 P de72cfe5d9384ef6906ffdb72de75060 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:51:27.571017 461 raft_consensus.cc:504] T 2c85626eb2574e8a80cd48c3d031fa51 P de72cfe5d9384ef6906ffdb72de75060 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2603ff4776e4e348f4bfbd67f20014e" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 44271 } } peers { permanent_uuid: "034fa62af1634ee4b1877650bf94b9b2" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 38637 } } peers { permanent_uuid: "de72cfe5d9384ef6906ffdb72de75060" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 39555 } }
I20260807 08:51:27.571136 461 leader_election.cc:288] T 2c85626eb2574e8a80cd48c3d031fa51 P de72cfe5d9384ef6906ffdb72de75060 [CANDIDATE]: Term 1 election: Requested vote from peers b2603ff4776e4e348f4bfbd67f20014e (127.4.159.129:44271), 034fa62af1634ee4b1877650bf94b9b2 (127.4.159.131:38637)
I20260807 08:51:27.571272 32419 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c85626eb2574e8a80cd48c3d031fa51" candidate_uuid: "de72cfe5d9384ef6906ffdb72de75060" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2603ff4776e4e348f4bfbd67f20014e"
I20260807 08:51:27.571341 355 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c85626eb2574e8a80cd48c3d031fa51" candidate_uuid: "de72cfe5d9384ef6906ffdb72de75060" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "034fa62af1634ee4b1877650bf94b9b2"
I20260807 08:51:27.571357 32419 raft_consensus.cc:3037] T 2c85626eb2574e8a80cd48c3d031fa51 P b2603ff4776e4e348f4bfbd67f20014e [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:51:27.571414 355 raft_consensus.cc:3037] T 2c85626eb2574e8a80cd48c3d031fa51 P 034fa62af1634ee4b1877650bf94b9b2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:51:27.572212 32419 raft_consensus.cc:2454] T 2c85626eb2574e8a80cd48c3d031fa51 P b2603ff4776e4e348f4bfbd67f20014e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de72cfe5d9384ef6906ffdb72de75060 in term 1.
I20260807 08:51:27.572212 355 raft_consensus.cc:2454] T 2c85626eb2574e8a80cd48c3d031fa51 P 034fa62af1634ee4b1877650bf94b9b2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de72cfe5d9384ef6906ffdb72de75060 in term 1.
I20260807 08:51:27.572382 32517 leader_election.cc:302] T 2c85626eb2574e8a80cd48c3d031fa51 P de72cfe5d9384ef6906ffdb72de75060 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2603ff4776e4e348f4bfbd67f20014e, de72cfe5d9384ef6906ffdb72de75060; no voters:
I20260807 08:51:27.572491 461 raft_consensus.cc:2781] T 2c85626eb2574e8a80cd48c3d031fa51 P de72cfe5d9384ef6906ffdb72de75060 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:51:27.572623 461 raft_consensus.cc:686] T 2c85626eb2574e8a80cd48c3d031fa51 P de72cfe5d9384ef6906ffdb72de75060 [term 1 LEADER]: Becoming Leader. State: Replica: de72cfe5d9384ef6906ffdb72de75060, State: Running, Role: LEADER
I20260807 08:51:27.572707 461 consensus_queue.cc:237] T 2c85626eb2574e8a80cd48c3d031fa51 P de72cfe5d9384ef6906ffdb72de75060 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2603ff4776e4e348f4bfbd67f20014e" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 44271 } } peers { permanent_uuid: "034fa62af1634ee4b1877650bf94b9b2" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 38637 } } peers { permanent_uuid: "de72cfe5d9384ef6906ffdb72de75060" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 39555 } }
I20260807 08:51:27.573493 32192 catalog_manager.cc:5495] T 2c85626eb2574e8a80cd48c3d031fa51 P de72cfe5d9384ef6906ffdb72de75060 reported cstate change: term changed from 0 to 1, leader changed from <none> to de72cfe5d9384ef6906ffdb72de75060 (127.4.159.130). New cstate: current_term: 1 leader_uuid: "de72cfe5d9384ef6906ffdb72de75060" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2603ff4776e4e348f4bfbd67f20014e" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 44271 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "034fa62af1634ee4b1877650bf94b9b2" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 38637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de72cfe5d9384ef6906ffdb72de75060" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 39555 } health_report { overall_health: HEALTHY } } }
W20260807 08:51:27.575733 32488 tablet.cc:2369] T 2c85626eb2574e8a80cd48c3d031fa51 P b2603ff4776e4e348f4bfbd67f20014e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 08:51:27.618539 429 tablet.cc:2369] T 2c85626eb2574e8a80cd48c3d031fa51 P 034fa62af1634ee4b1877650bf94b9b2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 08:51:27.622815 447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:27.620499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37556 (local address 127.4.159.130:39555)
0807 08:51:27.620558 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:27.620561 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:27.620584 (+ 23us) server_negotiation.cc:407] Connection header received
0807 08:51:27.620619 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:27.620620 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:27.620635 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:27.620690 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:51:27.620833 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:27.621310 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:27.622058 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:27.622249 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:27.622302 (+ 53us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:27.622488 (+ 186us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:51:27.622578 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:51:27.622660 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:27.622715 (+ 55us) server_negotiation.cc:299] Negotiation successful
0807 08:51:27.622725 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260807 08:51:27.624406 355 raft_consensus.cc:1261] T 2c85626eb2574e8a80cd48c3d031fa51 P 034fa62af1634ee4b1877650bf94b9b2 [term 1 FOLLOWER]: Refusing update from remote peer de72cfe5d9384ef6906ffdb72de75060: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:51:27.624408 32419 raft_consensus.cc:1261] T 2c85626eb2574e8a80cd48c3d031fa51 P b2603ff4776e4e348f4bfbd67f20014e [term 1 FOLLOWER]: Refusing update from remote peer de72cfe5d9384ef6906ffdb72de75060: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:51:27.624744 464 consensus_queue.cc:1035] T 2c85626eb2574e8a80cd48c3d031fa51 P de72cfe5d9384ef6906ffdb72de75060 [LEADER]: Connected to new peer: Peer: permanent_uuid: "034fa62af1634ee4b1877650bf94b9b2" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 38637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:51:27.624864 461 consensus_queue.cc:1035] T 2c85626eb2574e8a80cd48c3d031fa51 P de72cfe5d9384ef6906ffdb72de75060 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2603ff4776e4e348f4bfbd67f20014e" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 44271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:51:27.626785 467 mvcc.cc:205] Tried to move back new op lower bound from 7315835648507195392 to 7315835648298283008. Current Snapshot: MvccSnapshot[applied={T|T < 7315835648507195392}]
I20260807 08:51:27.627178 469 mvcc.cc:205] Tried to move back new op lower bound from 7315835648507195392 to 7315835648298283008. Current Snapshot: MvccSnapshot[applied={T|T < 7315835648507195392}]
I20260807 08:51:27.628782 472 mvcc.cc:205] Tried to move back new op lower bound from 7315835648507195392 to 7315835648298283008. Current Snapshot: MvccSnapshot[applied={T|T < 7315835648507195392}]
I20260807 08:51:27.629215 32195 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45626:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:51:27.629308 32195 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45626:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:51:27.630028 32195 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c645265f895c4952bd7911cdc193a00b: Sending DeleteTablet for 3 replicas of tablet 2c85626eb2574e8a80cd48c3d031fa51
I20260807 08:51:27.630374 32590 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c85626eb2574e8a80cd48c3d031fa51 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:51:27 UTC) from {username='kudu', principal='kudu/127.4.159.190@KRBTEST.COM'} at 127.0.0.1:37554
I20260807 08:51:27.630519 32383 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c85626eb2574e8a80cd48c3d031fa51 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:51:27 UTC) from {username='kudu', principal='kudu/127.4.159.190@KRBTEST.COM'} at 127.0.0.1:37832
I20260807 08:51:27.630564 318 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c85626eb2574e8a80cd48c3d031fa51 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:51:27 UTC) from {username='kudu', principal='kudu/127.4.159.190@KRBTEST.COM'} at 127.0.0.1:44626
I20260807 08:51:27.630745 479 tablet_replica.cc:330] T 2c85626eb2574e8a80cd48c3d031fa51 P de72cfe5d9384ef6906ffdb72de75060: stopping tablet replica
I20260807 08:51:27.630833 479 raft_consensus.cc:2229] T 2c85626eb2574e8a80cd48c3d031fa51 P de72cfe5d9384ef6906ffdb72de75060 [term 1 LEADER]: Raft consensus shutting down.
I20260807 08:51:27.630916 479 raft_consensus.cc:2258] T 2c85626eb2574e8a80cd48c3d031fa51 P de72cfe5d9384ef6906ffdb72de75060 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:51:27.630918 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 32282
I20260807 08:51:27.633392 479 ts_tablet_manager.cc:1905] T 2c85626eb2574e8a80cd48c3d031fa51 P de72cfe5d9384ef6906ffdb72de75060: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:51:27.635027 479 ts_tablet_manager.cc:1918] T 2c85626eb2574e8a80cd48c3d031fa51 P de72cfe5d9384ef6906ffdb72de75060: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:51:27.635092 479 log.cc:1196] T 2c85626eb2574e8a80cd48c3d031fa51 P de72cfe5d9384ef6906ffdb72de75060: Deleting WAL directory at /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786092453486433-4734-0/minicluster-data/ts-1/wal/wals/2c85626eb2574e8a80cd48c3d031fa51
I20260807 08:51:27.635371 479 ts_tablet_manager.cc:1939] T 2c85626eb2574e8a80cd48c3d031fa51 P de72cfe5d9384ef6906ffdb72de75060: Deleting consensus metadata
I20260807 08:51:27.635774 32171 catalog_manager.cc:4841] TS de72cfe5d9384ef6906ffdb72de75060 (127.4.159.130:39555): tablet 2c85626eb2574e8a80cd48c3d031fa51 (table test-table [id=6b727daa71034b7faae9e4fce3886d4d]) successfully deleted
I20260807 08:51:27.648353 481 tablet_replica.cc:330] T 2c85626eb2574e8a80cd48c3d031fa51 P 034fa62af1634ee4b1877650bf94b9b2: stopping tablet replica
I20260807 08:51:27.648456 481 raft_consensus.cc:2229] T 2c85626eb2574e8a80cd48c3d031fa51 P 034fa62af1634ee4b1877650bf94b9b2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:51:27.648554 481 raft_consensus.cc:2258] T 2c85626eb2574e8a80cd48c3d031fa51 P 034fa62af1634ee4b1877650bf94b9b2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:51:27.650638 481 ts_tablet_manager.cc:1905] T 2c85626eb2574e8a80cd48c3d031fa51 P 034fa62af1634ee4b1877650bf94b9b2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:51:27.651873 481 ts_tablet_manager.cc:1918] T 2c85626eb2574e8a80cd48c3d031fa51 P 034fa62af1634ee4b1877650bf94b9b2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:51:27.651938 481 log.cc:1196] T 2c85626eb2574e8a80cd48c3d031fa51 P 034fa62af1634ee4b1877650bf94b9b2: Deleting WAL directory at /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786092453486433-4734-0/minicluster-data/ts-2/wal/wals/2c85626eb2574e8a80cd48c3d031fa51
I20260807 08:51:27.652199 481 ts_tablet_manager.cc:1939] T 2c85626eb2574e8a80cd48c3d031fa51 P 034fa62af1634ee4b1877650bf94b9b2: Deleting consensus metadata
I20260807 08:51:27.652552 32175 catalog_manager.cc:4841] TS 034fa62af1634ee4b1877650bf94b9b2 (127.4.159.131:38637): tablet 2c85626eb2574e8a80cd48c3d031fa51 (table test-table [id=6b727daa71034b7faae9e4fce3886d4d]) successfully deleted
W20260807 08:51:27.653559 32170 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.159.129:44271 (error 108)
I20260807 08:51:27.653812 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 32492
I20260807 08:51:27.662803 442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:27.662660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.129:44271 (local address 127.0.0.1:37834)
0807 08:51:27.662712 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:27.662769 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.129:44271: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260807 08:51:27.662909 32170 catalog_manager.cc:4584] TS b2603ff4776e4e348f4bfbd67f20014e (127.4.159.129:44271): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2c85626eb2574e8a80cd48c3d031fa51: Network error: Client connection negotiation failed: client connection to 127.4.159.129:44271: connect: Connection refused (error 111)
I20260807 08:51:27.670202 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 32700
I20260807 08:51:27.678050 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 32146
2026-08-07T08:51:27Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2127 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-07T08:51:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:51:27Z Disabled control of system clock
I20260807 08:51:27.705494 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786092453486433-4734-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786092453486433-4734-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786092453486433-4734-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.159.190:35953
--webserver_interface=127.4.159.190
--webserver_port=0
--builtin_ntp_servers=127.4.159.148:41197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.159.190:35953
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 08:51:27.783900 487 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:51:27.784096 487 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:51:27.784121 487 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:51:27.785203 487 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:51:27.785248 487 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:51:27.785266 487 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:51:27.785282 487 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:51:27.785305 487 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:51:27.786626 487 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:41197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786092453486433-4734-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786092453486433-4734-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.159.190:35953
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.159.190:35953
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.159.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.487
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786092453486433-4734-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:51:27.786844 487 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:51:27.787084 487 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:51:27.791446 496 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:51:27.791502 500 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:51:27.791460 487 server_base.cc:1034] running on GCE node
W20260807 08:51:27.791471 506 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:51:27.791865 487 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:51:27.792121 487 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:51:27.793263 487 hybrid_clock.cc:648] HybridClock initialized: now 1786092687793251 us; error 27 us; skew 500 ppm
I20260807 08:51:27.794364 487 webserver.cc:460] Webserver started at http://127.4.159.190:34567/ using document root <none> and password file <none>
I20260807 08:51:27.794572 487 fs_manager.cc:359] Metadata directory not provided
I20260807 08:51:27.794617 487 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:51:27.794744 487 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:51:27.795701 487 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786092453486433-4734-0/minicluster-data/master-0/data/instance:
uuid: "eab89cccc4684dffa92ac33f8dbb9334"
format_stamp: "Formatted at 2026-08-07 08:51:27 on dist-test-slave-6zbq"
server_key: "3872161a2f894446f2c11afd905de468"
server_key_iv: "13628534b00c29e1f94ea160f7cb0715"
server_key_version: "encryptionkey@0"
I20260807 08:51:27.796017 487 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786092453486433-4734-0/minicluster-data/master-0/wal/instance:
uuid: "eab89cccc4684dffa92ac33f8dbb9334"
format_stamp: "Formatted at 2026-08-07 08:51:27 on dist-test-slave-6zbq"
server_key: "3872161a2f894446f2c11afd905de468"
server_key_iv: "13628534b00c29e1f94ea160f7cb0715"
server_key_version: "encryptionkey@0"
I20260807 08:51:27.797155 487 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 08:51:27.797823 513 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:51:27.797969 487 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 08:51:27.798048 487 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786092453486433-4734-0/minicluster-data/master-0/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786092453486433-4734-0/minicluster-data/master-0/wal
uuid: "eab89cccc4684dffa92ac33f8dbb9334"
format_stamp: "Formatted at 2026-08-07 08:51:27 on dist-test-slave-6zbq"
server_key: "3872161a2f894446f2c11afd905de468"
server_key_iv: "13628534b00c29e1f94ea160f7cb0715"
server_key_version: "encryptionkey@0"
I20260807 08:51:27.798110 487 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786092453486433-4734-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786092453486433-4734-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786092453486433-4734-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:51:27.819720 487 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:51:27.819983 487 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:51:27.820103 487 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:51:27.823712 487 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.190:35953
I20260807 08:51:27.823743 595 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.190:35953 every 8 connection(s)
I20260807 08:51:27.824026 487 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
I20260807 08:51:27.824891 596 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:51:27.827019 596 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eab89cccc4684dffa92ac33f8dbb9334: Bootstrap starting.
I20260807 08:51:27.827543 596 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eab89cccc4684dffa92ac33f8dbb9334: Neither blocks nor log segments found. Creating new log.
I20260807 08:51:27.827807 596 log.cc:824] T 00000000000000000000000000000000 P eab89cccc4684dffa92ac33f8dbb9334: Log is configured to *not* fsync() on all Append() calls
I20260807 08:51:27.828377 596 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eab89cccc4684dffa92ac33f8dbb9334: No bootstrap required, opened a new log
I20260807 08:51:27.829473 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 487
I20260807 08:51:27.829586 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786092453486433-4734-0/minicluster-data/master-0/wal/instance
I20260807 08:51:27.829764 4734 external_mini_cluster.cc:1428] Setting key 12583c3005a36e6cd8eb30d7ba77ce42
I20260807 08:51:27.830163 596 raft_consensus.cc:348] T 00000000000000000000000000000000 P eab89cccc4684dffa92ac33f8dbb9334 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eab89cccc4684dffa92ac33f8dbb9334" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 35953 } }
I20260807 08:51:27.830291 596 raft_consensus.cc:374] T 00000000000000000000000000000000 P eab89cccc4684dffa92ac33f8dbb9334 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:51:27.830322 596 raft_consensus.cc:729] T 00000000000000000000000000000000 P eab89cccc4684dffa92ac33f8dbb9334 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eab89cccc4684dffa92ac33f8dbb9334, State: Initialized, Role: FOLLOWER
I20260807 08:51:27.830470 596 consensus_queue.cc:260] T 00000000000000000000000000000000 P eab89cccc4684dffa92ac33f8dbb9334 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eab89cccc4684dffa92ac33f8dbb9334" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 35953 } }
I20260807 08:51:27.830541 596 raft_consensus.cc:388] T 00000000000000000000000000000000 P eab89cccc4684dffa92ac33f8dbb9334 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:51:27.830572 596 raft_consensus.cc:482] T 00000000000000000000000000000000 P eab89cccc4684dffa92ac33f8dbb9334 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:51:27.830624 596 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eab89cccc4684dffa92ac33f8dbb9334 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:51:27.831092 596 raft_consensus.cc:504] T 00000000000000000000000000000000 P eab89cccc4684dffa92ac33f8dbb9334 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eab89cccc4684dffa92ac33f8dbb9334" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 35953 } }
I20260807 08:51:27.831184 596 leader_election.cc:302] T 00000000000000000000000000000000 P eab89cccc4684dffa92ac33f8dbb9334 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eab89cccc4684dffa92ac33f8dbb9334; no voters:
I20260807 08:51:27.831323 596 leader_election.cc:288] T 00000000000000000000000000000000 P eab89cccc4684dffa92ac33f8dbb9334 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:51:27.831369 603 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eab89cccc4684dffa92ac33f8dbb9334 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:51:27.831506 603 raft_consensus.cc:686] T 00000000000000000000000000000000 P eab89cccc4684dffa92ac33f8dbb9334 [term 1 LEADER]: Becoming Leader. State: Replica: eab89cccc4684dffa92ac33f8dbb9334, State: Running, Role: LEADER
I20260807 08:51:27.831547 596 sys_catalog.cc:564] T 00000000000000000000000000000000 P eab89cccc4684dffa92ac33f8dbb9334 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:51:27.831589 603 consensus_queue.cc:237] T 00000000000000000000000000000000 P eab89cccc4684dffa92ac33f8dbb9334 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eab89cccc4684dffa92ac33f8dbb9334" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 35953 } }
I20260807 08:51:27.831923 603 sys_catalog.cc:455] T 00000000000000000000000000000000 P eab89cccc4684dffa92ac33f8dbb9334 [sys.catalog]: SysCatalogTable state changed. Reason: New leader eab89cccc4684dffa92ac33f8dbb9334. Latest consensus state: current_term: 1 leader_uuid: "eab89cccc4684dffa92ac33f8dbb9334" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eab89cccc4684dffa92ac33f8dbb9334" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 35953 } } }
I20260807 08:51:27.832000 603 sys_catalog.cc:458] T 00000000000000000000000000000000 P eab89cccc4684dffa92ac33f8dbb9334 [sys.catalog]: This master's current role is: LEADER
I20260807 08:51:27.831939 604 sys_catalog.cc:455] T 00000000000000000000000000000000 P eab89cccc4684dffa92ac33f8dbb9334 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eab89cccc4684dffa92ac33f8dbb9334" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eab89cccc4684dffa92ac33f8dbb9334" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 35953 } } }
I20260807 08:51:27.832046 604 sys_catalog.cc:458] T 00000000000000000000000000000000 P eab89cccc4684dffa92ac33f8dbb9334 [sys.catalog]: This master's current role is: LEADER
I20260807 08:51:27.832926 611 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:51:27.833357 611 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:51:27.835047 611 catalog_manager.cc:1347] Generated new cluster ID: 1cbe3287269a46f887a393cd4fbb7096
I20260807 08:51:27.835095 611 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:51:27.835685 600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:27.830226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49996 (local address 127.4.159.190:35953)
0807 08:51:27.830446 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:27.830455 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:27.830483 (+ 28us) server_negotiation.cc:407] Connection header received
0807 08:51:27.830578 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:27.830585 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:27.830649 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:27.830794 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:51:27.830997 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:27.832558 (+ 1561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:27.833595 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:27.834189 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:27.834310 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:27.834903 (+ 593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:51:27.834907 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:51:27.834907 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:51:27.835092 (+ 185us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:51:27.835142 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:51:27.835156 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:51:27.835223 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:27.835415 (+ 192us) server_negotiation.cc:299] Negotiation successful
0807 08:51:27.835487 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":109,"threads_started":1}
I20260807 08:51:27.852946 611 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:51:27.853406 611 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:51:27.859550 611 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eab89cccc4684dffa92ac33f8dbb9334: Generated new TSK 0
I20260807 08:51:27.859679 611 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:51:27.867551 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786092453486433-4734-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786092453486433-4734-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786092453486433-4734-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786092453486433-4734-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.159.129:0
--local_ip_for_outbound_sockets=127.4.159.129
--webserver_interface=127.4.159.129
--webserver_port=0
--tserver_master_addrs=127.4.159.190:35953
--builtin_ntp_servers=127.4.159.148:41197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 08:51:27.947204 630 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:51:27.947395 630 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:51:27.947417 630 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:51:27.948537 630 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:51:27.948572 630 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:51:27.948596 630 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.159.129
I20260807 08:51:27.949872 630 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:41197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786092453486433-4734-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786092453486433-4734-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.159.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786092453486433-4734-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.159.129
--webserver_port=0
--tserver_master_addrs=127.4.159.190:35953
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.630
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.159.129
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786092453486433-4734-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:51:27.950085 630 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:51:27.950376 630 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 08:51:27.954959 630 server_base.cc:1034] running on GCE node
W20260807 08:51:27.954936 637 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:51:27.954928 642 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:51:27.954911 638 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:51:27.955286 630 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:51:27.955502 630 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:51:27.956634 630 hybrid_clock.cc:648] HybridClock initialized: now 1786092687956625 us; error 28 us; skew 500 ppm
I20260807 08:51:27.957806 630 webserver.cc:460] Webserver started at http://127.4.159.129:35355/ using document root <none> and password file <none>
I20260807 08:51:27.958009 630 fs_manager.cc:359] Metadata directory not provided
I20260807 08:51:27.958051 630 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:51:27.958171 630 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:51:27.959178 630 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786092453486433-4734-0/minicluster-data/ts-0/data/instance:
uuid: "b0b7aa7bc88d4b08875d1f07d200ef0e"
format_stamp: "Formatted at 2026-08-07 08:51:27 on dist-test-slave-6zbq"
server_key: "e9aebbaa1708137d249e633840771580"
server_key_iv: "39f08b9a7c0d4de89a2bf009304495c5"
server_key_version: "encryptionkey@0"
I20260807 08:51:27.959496 630 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786092453486433-4734-0/minicluster-data/ts-0/wal/instance:
uuid: "b0b7aa7bc88d4b08875d1f07d200ef0e"
format_stamp: "Formatted at 2026-08-07 08:51:27 on dist-test-slave-6zbq"
server_key: "e9aebbaa1708137d249e633840771580"
server_key_iv: "39f08b9a7c0d4de89a2bf009304495c5"
server_key_version: "encryptionkey@0"
I20260807 08:51:27.960644 630 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 08:51:27.961246 652 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:51:27.961397 630 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 08:51:27.961476 630 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786092453486433-4734-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786092453486433-4734-0/minicluster-data/ts-0/wal
uuid: "b0b7aa7bc88d4b08875d1f07d200ef0e"
format_stamp: "Formatted at 2026-08-07 08:51:27 on dist-test-slave-6zbq"
server_key: "e9aebbaa1708137d249e633840771580"
server_key_iv: "39f08b9a7c0d4de89a2bf009304495c5"
server_key_version: "encryptionkey@0"
I20260807 08:51:27.961541 630 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786092453486433-4734-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786092453486433-4734-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786092453486433-4734-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:51:27.977275 630 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:51:27.977502 630 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:51:27.977613 630 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:51:27.977797 630 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:51:27.978075 630 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:51:27.978108 630 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:51:27.978132 630 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:51:27.978157 630 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:51:27.983304 630 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.129:39809
I20260807 08:51:27.983340 822 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.129:39809 every 8 connection(s)
I20260807 08:51:27.983605 630 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786092453486433-4734-0/minicluster-data/ts-0/data/info.pb
I20260807 08:51:27.987975 600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:27.984211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.129:35429 (local address 127.4.159.190:35953)
0807 08:51:27.984252 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:27.984255 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:27.984453 (+ 198us) server_negotiation.cc:407] Connection header received
0807 08:51:27.984860 (+ 407us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:27.984861 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:27.984870 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:27.984921 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:51:27.985520 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:27.985997 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:27.986915 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:27.987094 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:27.987156 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:27.987775 (+ 619us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:51:27.987776 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:51:27.987777 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:51:27.987781 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:51:27.987794 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:51:27.987798 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:51:27.987837 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:27.987895 (+ 58us) server_negotiation.cc:299] Negotiation successful
0807 08:51:27.987920 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260807 08:51:27.988142 828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:27.984175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:35953 (local address 127.4.159.129:35429)
0807 08:51:27.984348 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:27.984383 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0807 08:51:27.984789 (+ 406us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:51:27.984984 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:51:27.984987 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:51:27.984995 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:51:27.985459 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:27.985463 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:27.986060 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:27.986060 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:27.986850 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:27.986853 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:27.987167 (+ 314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:27.987167 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:27.987273 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:27.987661 (+ 388us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 08:51:27.987662 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:51:27.987686 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 08:51:27.987706 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:51:27.987842 (+ 136us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:51:27.987844 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:51:27.987847 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 08:51:27.987889 (+ 42us) client_negotiation.cc:240] Negotiation successful
0807 08:51:27.987958 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":66,"threads_started":1}
I20260807 08:51:27.988575 825 heartbeater.cc:347] Connected to a master server at 127.4.159.190:35953
I20260807 08:51:27.988665 825 heartbeater.cc:464] Registering TS with master...
I20260807 08:51:27.988802 825 heartbeater.cc:511] Master 127.4.159.190:35953 requested a full tablet report, sending...
I20260807 08:51:27.989264 541 ts_manager.cc:194] Registered new tserver with Master: b0b7aa7bc88d4b08875d1f07d200ef0e (127.4.159.129:39809)
I20260807 08:51:27.989796 541 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.159.129:35429
I20260807 08:51:27.992601 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 630
I20260807 08:51:27.992677 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786092453486433-4734-0/minicluster-data/ts-0/wal/instance
I20260807 08:51:27.992825 4734 external_mini_cluster.cc:1428] Setting key c38491803d2239570eb449126a5d3faa
I20260807 08:51:27.993579 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786092453486433-4734-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786092453486433-4734-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786092453486433-4734-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786092453486433-4734-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.159.130:0
--local_ip_for_outbound_sockets=127.4.159.130
--webserver_interface=127.4.159.130
--webserver_port=0
--tserver_master_addrs=127.4.159.190:35953
--builtin_ntp_servers=127.4.159.148:41197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 08:51:28.071918 829 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:51:28.072093 829 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:51:28.072111 829 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:51:28.073184 829 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:51:28.073221 829 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:51:28.073248 829 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.159.130
I20260807 08:51:28.074566 829 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:41197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786092453486433-4734-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786092453486433-4734-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.159.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786092453486433-4734-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.159.130
--webserver_port=0
--tserver_master_addrs=127.4.159.190:35953
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.829
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.159.130
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786092453486433-4734-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:51:28.074765 829 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:51:28.075011 829 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:51:28.079406 841 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:51:28.079407 834 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:51:28.079427 836 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:51:28.079545 829 server_base.cc:1034] running on GCE node
I20260807 08:51:28.079774 829 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:51:28.079990 829 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:51:28.081144 829 hybrid_clock.cc:648] HybridClock initialized: now 1786092688081131 us; error 27 us; skew 500 ppm
I20260807 08:51:28.082307 829 webserver.cc:460] Webserver started at http://127.4.159.130:38573/ using document root <none> and password file <none>
I20260807 08:51:28.082527 829 fs_manager.cc:359] Metadata directory not provided
I20260807 08:51:28.082576 829 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:51:28.082703 829 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:51:28.083657 829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786092453486433-4734-0/minicluster-data/ts-1/data/instance:
uuid: "98ad6bb1a58c4061ac1c40ade9b3c9ac"
format_stamp: "Formatted at 2026-08-07 08:51:28 on dist-test-slave-6zbq"
server_key: "629ad6d11615df369212853e4e1d175a"
server_key_iv: "da901749082cdde8e8ac1dca661297c8"
server_key_version: "encryptionkey@0"
I20260807 08:51:28.083966 829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786092453486433-4734-0/minicluster-data/ts-1/wal/instance:
uuid: "98ad6bb1a58c4061ac1c40ade9b3c9ac"
format_stamp: "Formatted at 2026-08-07 08:51:28 on dist-test-slave-6zbq"
server_key: "629ad6d11615df369212853e4e1d175a"
server_key_iv: "da901749082cdde8e8ac1dca661297c8"
server_key_version: "encryptionkey@0"
I20260807 08:51:28.085093 829 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 08:51:28.085709 851 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:51:28.085858 829 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 08:51:28.085937 829 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786092453486433-4734-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786092453486433-4734-0/minicluster-data/ts-1/wal
uuid: "98ad6bb1a58c4061ac1c40ade9b3c9ac"
format_stamp: "Formatted at 2026-08-07 08:51:28 on dist-test-slave-6zbq"
server_key: "629ad6d11615df369212853e4e1d175a"
server_key_iv: "da901749082cdde8e8ac1dca661297c8"
server_key_version: "encryptionkey@0"
I20260807 08:51:28.085999 829 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786092453486433-4734-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786092453486433-4734-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786092453486433-4734-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:51:28.094960 829 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:51:28.095192 829 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:51:28.095301 829 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:51:28.095477 829 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:51:28.095750 829 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:51:28.095782 829 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:51:28.095806 829 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:51:28.095831 829 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:51:28.100880 829 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.130:45753
I20260807 08:51:28.100941 1008 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.130:45753 every 8 connection(s)
I20260807 08:51:28.101243 829 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786092453486433-4734-0/minicluster-data/ts-1/data/info.pb
I20260807 08:51:28.105911 600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:28.101828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.130:45571 (local address 127.4.159.190:35953)
0807 08:51:28.101867 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:28.101869 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:28.102112 (+ 243us) server_negotiation.cc:407] Connection header received
0807 08:51:28.102600 (+ 488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:28.102601 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:28.102612 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:28.102662 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:51:28.103246 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:28.103740 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:28.104781 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:28.104971 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:28.105025 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:28.105713 (+ 688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:51:28.105715 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:51:28.105715 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:51:28.105719 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:51:28.105733 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:51:28.105737 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:51:28.105777 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:28.105822 (+ 45us) server_negotiation.cc:299] Negotiation successful
0807 08:51:28.105847 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260807 08:51:28.106088 1011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:28.101811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:35953 (local address 127.4.159.130:45571)
0807 08:51:28.101982 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:28.102011 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0807 08:51:28.102495 (+ 484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:51:28.102710 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:51:28.102713 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:51:28.102721 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:51:28.103159 (+ 438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:28.103163 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:28.103909 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:28.103911 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:28.104700 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:28.104703 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:28.105034 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:28.105035 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:28.105133 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:28.105581 (+ 448us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 08:51:28.105582 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:51:28.105618 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 08:51:28.105640 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:51:28.105779 (+ 139us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:51:28.105781 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:51:28.105783 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 08:51:28.105823 (+ 40us) client_negotiation.cc:240] Negotiation successful
0807 08:51:28.105900 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":89,"threads_started":1}
I20260807 08:51:28.106572 1009 heartbeater.cc:347] Connected to a master server at 127.4.159.190:35953
I20260807 08:51:28.106660 1009 heartbeater.cc:464] Registering TS with master...
I20260807 08:51:28.106817 1009 heartbeater.cc:511] Master 127.4.159.190:35953 requested a full tablet report, sending...
I20260807 08:51:28.107175 541 ts_manager.cc:194] Registered new tserver with Master: 98ad6bb1a58c4061ac1c40ade9b3c9ac (127.4.159.130:45753)
I20260807 08:51:28.107499 541 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.159.130:45571
I20260807 08:51:28.108246 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 829
I20260807 08:51:28.108323 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786092453486433-4734-0/minicluster-data/ts-1/wal/instance
I20260807 08:51:28.108462 4734 external_mini_cluster.cc:1428] Setting key 48b0fcfb3c3ff51cb838af1464373d70
I20260807 08:51:28.109207 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786092453486433-4734-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786092453486433-4734-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786092453486433-4734-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786092453486433-4734-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.159.131:0
--local_ip_for_outbound_sockets=127.4.159.131
--webserver_interface=127.4.159.131
--webserver_port=0
--tserver_master_addrs=127.4.159.190:35953
--builtin_ntp_servers=127.4.159.148:41197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 08:51:28.189373 1013 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:51:28.189553 1013 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:51:28.189572 1013 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:51:28.190713 1013 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:51:28.190750 1013 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:51:28.190778 1013 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.159.131
I20260807 08:51:28.192085 1013 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:41197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786092453486433-4734-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786092453486433-4734-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.159.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786092453486433-4734-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.159.131
--webserver_port=0
--tserver_master_addrs=127.4.159.190:35953
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1013
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.159.131
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786092453486433-4734-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:51:28.192282 1013 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:51:28.192533 1013 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:51:28.197016 1025 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:51:28.197045 1020 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:51:28.197089 1013 server_base.cc:1034] running on GCE node
W20260807 08:51:28.197048 1022 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:51:28.197403 1013 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:51:28.197595 1013 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:51:28.198722 1013 hybrid_clock.cc:648] HybridClock initialized: now 1786092688198708 us; error 27 us; skew 500 ppm
I20260807 08:51:28.199834 1013 webserver.cc:460] Webserver started at http://127.4.159.131:42155/ using document root <none> and password file <none>
I20260807 08:51:28.200055 1013 fs_manager.cc:359] Metadata directory not provided
I20260807 08:51:28.200101 1013 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:51:28.200232 1013 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:51:28.201206 1013 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786092453486433-4734-0/minicluster-data/ts-2/data/instance:
uuid: "3fff30b4948b409995102937d8eca802"
format_stamp: "Formatted at 2026-08-07 08:51:28 on dist-test-slave-6zbq"
server_key: "080f952cf95e91a2721bee6914dcb149"
server_key_iv: "7661b698cfb933cb4941446ae91af9b8"
server_key_version: "encryptionkey@0"
I20260807 08:51:28.201541 1013 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786092453486433-4734-0/minicluster-data/ts-2/wal/instance:
uuid: "3fff30b4948b409995102937d8eca802"
format_stamp: "Formatted at 2026-08-07 08:51:28 on dist-test-slave-6zbq"
server_key: "080f952cf95e91a2721bee6914dcb149"
server_key_iv: "7661b698cfb933cb4941446ae91af9b8"
server_key_version: "encryptionkey@0"
I20260807 08:51:28.202744 1013 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 08:51:28.203408 1034 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:51:28.203591 1013 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 08:51:28.203676 1013 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786092453486433-4734-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786092453486433-4734-0/minicluster-data/ts-2/wal
uuid: "3fff30b4948b409995102937d8eca802"
format_stamp: "Formatted at 2026-08-07 08:51:28 on dist-test-slave-6zbq"
server_key: "080f952cf95e91a2721bee6914dcb149"
server_key_iv: "7661b698cfb933cb4941446ae91af9b8"
server_key_version: "encryptionkey@0"
I20260807 08:51:28.203733 1013 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786092453486433-4734-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786092453486433-4734-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786092453486433-4734-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:51:28.225286 1013 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:51:28.225522 1013 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:51:28.225638 1013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:51:28.225826 1013 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:51:28.226104 1013 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:51:28.226135 1013 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:51:28.226161 1013 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:51:28.226176 1013 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:51:28.231252 1013 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.131:39911
I20260807 08:51:28.231312 1184 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.131:39911 every 8 connection(s)
I20260807 08:51:28.231604 1013 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786092453486433-4734-0/minicluster-data/ts-2/data/info.pb
I20260807 08:51:28.233989 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 1013
I20260807 08:51:28.234076 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786092453486433-4734-0/minicluster-data/ts-2/wal/instance
I20260807 08:51:28.234237 4734 external_mini_cluster.cc:1428] Setting key 2225bf06d374bb885831c4433ef69b63
I20260807 08:51:28.236503 600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:28.232275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.131:33089 (local address 127.4.159.190:35953)
0807 08:51:28.232335 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:28.232337 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:28.232579 (+ 242us) server_negotiation.cc:407] Connection header received
0807 08:51:28.233045 (+ 466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:28.233046 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:28.233056 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:28.233094 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:51:28.233680 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:28.234214 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:28.235433 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:28.235615 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:28.235667 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:28.236332 (+ 665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:51:28.236333 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:51:28.236333 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:51:28.236336 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:51:28.236345 (+ 9us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:51:28.236349 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:51:28.236381 (+ 32us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:28.236428 (+ 47us) server_negotiation.cc:299] Negotiation successful
0807 08:51:28.236448 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260807 08:51:28.236681 1196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:28.232255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:35953 (local address 127.4.159.131:33089)
0807 08:51:28.232431 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:28.232507 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0807 08:51:28.232962 (+ 455us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:51:28.233151 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:51:28.233154 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:51:28.233162 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:51:28.233613 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:28.233617 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:28.234304 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:28.234305 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:28.235364 (+ 1059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:28.235367 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:28.235676 (+ 309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:28.235676 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:28.235774 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:28.236221 (+ 447us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 08:51:28.236222 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:51:28.236249 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 08:51:28.236271 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:51:28.236384 (+ 113us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:51:28.236386 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:51:28.236388 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 08:51:28.236429 (+ 41us) client_negotiation.cc:240] Negotiation successful
0807 08:51:28.236499 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":80,"threads_started":1}
I20260807 08:51:28.237071 1189 heartbeater.cc:347] Connected to a master server at 127.4.159.190:35953
I20260807 08:51:28.237155 1189 heartbeater.cc:464] Registering TS with master...
I20260807 08:51:28.237287 1189 heartbeater.cc:511] Master 127.4.159.190:35953 requested a full tablet report, sending...
I20260807 08:51:28.237588 541 ts_manager.cc:194] Registered new tserver with Master: 3fff30b4948b409995102937d8eca802 (127.4.159.131:39911)
I20260807 08:51:28.237900 541 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.159.131:33089
I20260807 08:51:28.245721 4734 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 08:51:28.249270 600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:28.246961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49998 (local address 127.4.159.190:35953)
0807 08:51:28.247003 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:28.247005 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:28.247094 (+ 89us) server_negotiation.cc:407] Connection header received
0807 08:51:28.247132 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:28.247133 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:28.247142 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:28.247188 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:51:28.247297 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:28.247726 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:28.248384 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:28.248554 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:28.248602 (+ 48us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:28.249075 (+ 473us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:51:28.249082 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:51:28.249083 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:51:28.249087 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:51:28.249098 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:51:28.249102 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:51:28.249168 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:28.249213 (+ 45us) server_negotiation.cc:299] Negotiation successful
0807 08:51:28.249222 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260807 08:51:28.250574 541 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49998:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 08:51:28.251966 541 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:51:28.257589 1203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:28.253765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.130:45753 (local address 127.0.0.1:42002)
0807 08:51:28.254067 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:28.254107 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 08:51:28.254167 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:51:28.254321 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:51:28.254322 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:51:28.254324 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:51:28.254400 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:28.254401 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:28.255448 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:28.255449 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:28.256488 (+ 1039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:28.256491 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:28.257326 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:28.257327 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:28.257400 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:28.257413 (+ 13us) client_negotiation.cc:769] Sending connection context
0807 08:51:28.257475 (+ 62us) client_negotiation.cc:240] Negotiation successful
0807 08:51:28.257519 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":87,"threads_started":1}
I20260807 08:51:28.257602 1206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:28.253876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42002 (local address 127.4.159.130:45753)
0807 08:51:28.254174 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:28.254178 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:28.254189 (+ 11us) server_negotiation.cc:407] Connection header received
0807 08:51:28.254231 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:28.254233 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:28.254247 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:28.254313 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:51:28.254463 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:28.255382 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:28.256665 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:28.257257 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:28.257315 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:28.257402 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:28.257486 (+ 84us) server_negotiation.cc:299] Negotiation successful
0807 08:51:28.257523 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262,"thread_start_us":75,"threads_started":1}
I20260807 08:51:28.258129 1208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:28.253916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.131:39911 (local address 127.0.0.1:39616)
0807 08:51:28.254695 (+ 779us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:28.254705 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 08:51:28.254753 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:51:28.254958 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:51:28.254959 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:51:28.254961 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:51:28.254997 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:28.254998 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:28.256088 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:28.256089 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:28.256944 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:28.256947 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:28.257884 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:28.257884 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:28.257935 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:28.257937 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 08:51:28.257975 (+ 38us) client_negotiation.cc:240] Negotiation successful
0807 08:51:28.258016 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":763,"mutex_wait_us":2,"thread_start_us":68,"threads_started":1}
I20260807 08:51:28.258244 1209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:28.253967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39616 (local address 127.4.159.131:39911)
0807 08:51:28.254135 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:28.254140 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:28.254804 (+ 664us) server_negotiation.cc:407] Connection header received
0807 08:51:28.254840 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:28.254842 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:28.254855 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:28.254913 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:51:28.255068 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:28.256022 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:28.257019 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:28.257675 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:28.257743 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:28.258119 (+ 376us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:28.258144 (+ 25us) server_negotiation.cc:299] Negotiation successful
0807 08:51:28.258178 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":88,"threads_started":1}
I20260807 08:51:28.258283 1207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:28.253900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55242 (local address 127.4.159.129:39809)
0807 08:51:28.254453 (+ 553us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:28.254457 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:28.254466 (+ 9us) server_negotiation.cc:407] Connection header received
0807 08:51:28.254503 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:28.254505 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:28.254519 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:28.254596 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:51:28.254732 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:28.255791 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:28.256948 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:28.257675 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:28.257743 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:28.257830 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:28.258170 (+ 340us) server_negotiation.cc:299] Negotiation successful
0807 08:51:28.258212 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":513,"thread_start_us":79,"threads_started":1}
I20260807 08:51:28.258132 1205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:28.253727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.129:39809 (local address 127.0.0.1:55242)
0807 08:51:28.254057 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:28.254107 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 08:51:28.254169 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:51:28.254612 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:51:28.254613 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:51:28.254615 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:51:28.254651 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:28.254653 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:28.255875 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:28.255876 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:28.256875 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:28.256879 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:28.257866 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:28.257868 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:28.257933 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:28.257936 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 08:51:28.257982 (+ 46us) client_negotiation.cc:240] Negotiation successful
0807 08:51:28.258016 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"mutex_wait_us":3,"thread_start_us":62,"threads_started":1}
I20260807 08:51:28.258437 915 tablet_service.cc:1467] Processing CreateTablet for tablet 05ec4889745e4ef5ac45cf5167e40533 (DEFAULT_TABLE table=test-table [id=338a72be143641628531d4731e588d55]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:51:28.258791 915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05ec4889745e4ef5ac45cf5167e40533. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:51:28.258831 1097 tablet_service.cc:1467] Processing CreateTablet for tablet 05ec4889745e4ef5ac45cf5167e40533 (DEFAULT_TABLE table=test-table [id=338a72be143641628531d4731e588d55]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:51:28.259147 1097 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05ec4889745e4ef5ac45cf5167e40533. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:51:28.261214 1210 tablet_bootstrap.cc:492] T 05ec4889745e4ef5ac45cf5167e40533 P 98ad6bb1a58c4061ac1c40ade9b3c9ac: Bootstrap starting.
I20260807 08:51:28.261744 729 tablet_service.cc:1467] Processing CreateTablet for tablet 05ec4889745e4ef5ac45cf5167e40533 (DEFAULT_TABLE table=test-table [id=338a72be143641628531d4731e588d55]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:51:28.261896 1210 tablet_bootstrap.cc:654] T 05ec4889745e4ef5ac45cf5167e40533 P 98ad6bb1a58c4061ac1c40ade9b3c9ac: Neither blocks nor log segments found. Creating new log.
I20260807 08:51:28.262076 729 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05ec4889745e4ef5ac45cf5167e40533. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:51:28.262214 1210 log.cc:824] T 05ec4889745e4ef5ac45cf5167e40533 P 98ad6bb1a58c4061ac1c40ade9b3c9ac: Log is configured to *not* fsync() on all Append() calls
I20260807 08:51:28.262967 1210 tablet_bootstrap.cc:492] T 05ec4889745e4ef5ac45cf5167e40533 P 98ad6bb1a58c4061ac1c40ade9b3c9ac: No bootstrap required, opened a new log
I20260807 08:51:28.263036 1210 ts_tablet_manager.cc:1397] T 05ec4889745e4ef5ac45cf5167e40533 P 98ad6bb1a58c4061ac1c40ade9b3c9ac: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 08:51:28.265627 1214 tablet_bootstrap.cc:492] T 05ec4889745e4ef5ac45cf5167e40533 P b0b7aa7bc88d4b08875d1f07d200ef0e: Bootstrap starting.
I20260807 08:51:28.265677 1210 raft_consensus.cc:348] T 05ec4889745e4ef5ac45cf5167e40533 P 98ad6bb1a58c4061ac1c40ade9b3c9ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0b7aa7bc88d4b08875d1f07d200ef0e" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 39809 } } peers { permanent_uuid: "98ad6bb1a58c4061ac1c40ade9b3c9ac" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 45753 } } peers { permanent_uuid: "3fff30b4948b409995102937d8eca802" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 39911 } }
I20260807 08:51:28.265828 1210 raft_consensus.cc:374] T 05ec4889745e4ef5ac45cf5167e40533 P 98ad6bb1a58c4061ac1c40ade9b3c9ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:51:28.265874 1210 raft_consensus.cc:729] T 05ec4889745e4ef5ac45cf5167e40533 P 98ad6bb1a58c4061ac1c40ade9b3c9ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98ad6bb1a58c4061ac1c40ade9b3c9ac, State: Initialized, Role: FOLLOWER
I20260807 08:51:28.265992 1210 consensus_queue.cc:260] T 05ec4889745e4ef5ac45cf5167e40533 P 98ad6bb1a58c4061ac1c40ade9b3c9ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0b7aa7bc88d4b08875d1f07d200ef0e" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 39809 } } peers { permanent_uuid: "98ad6bb1a58c4061ac1c40ade9b3c9ac" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 45753 } } peers { permanent_uuid: "3fff30b4948b409995102937d8eca802" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 39911 } }
I20260807 08:51:28.266252 1214 tablet_bootstrap.cc:654] T 05ec4889745e4ef5ac45cf5167e40533 P b0b7aa7bc88d4b08875d1f07d200ef0e: Neither blocks nor log segments found. Creating new log.
I20260807 08:51:28.266245 1210 ts_tablet_manager.cc:1428] T 05ec4889745e4ef5ac45cf5167e40533 P 98ad6bb1a58c4061ac1c40ade9b3c9ac: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 08:51:28.266265 1009 heartbeater.cc:503] Master 127.4.159.190:35953 was elected leader, sending a full tablet report...
I20260807 08:51:28.266530 1214 log.cc:824] T 05ec4889745e4ef5ac45cf5167e40533 P b0b7aa7bc88d4b08875d1f07d200ef0e: Log is configured to *not* fsync() on all Append() calls
I20260807 08:51:28.266836 1212 tablet_bootstrap.cc:492] T 05ec4889745e4ef5ac45cf5167e40533 P 3fff30b4948b409995102937d8eca802: Bootstrap starting.
I20260807 08:51:28.267349 1212 tablet_bootstrap.cc:654] T 05ec4889745e4ef5ac45cf5167e40533 P 3fff30b4948b409995102937d8eca802: Neither blocks nor log segments found. Creating new log.
I20260807 08:51:28.267596 1212 log.cc:824] T 05ec4889745e4ef5ac45cf5167e40533 P 3fff30b4948b409995102937d8eca802: Log is configured to *not* fsync() on all Append() calls
I20260807 08:51:28.268024 1214 tablet_bootstrap.cc:492] T 05ec4889745e4ef5ac45cf5167e40533 P b0b7aa7bc88d4b08875d1f07d200ef0e: No bootstrap required, opened a new log
I20260807 08:51:28.268095 1214 ts_tablet_manager.cc:1397] T 05ec4889745e4ef5ac45cf5167e40533 P b0b7aa7bc88d4b08875d1f07d200ef0e: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260807 08:51:28.268227 1212 tablet_bootstrap.cc:492] T 05ec4889745e4ef5ac45cf5167e40533 P 3fff30b4948b409995102937d8eca802: No bootstrap required, opened a new log
I20260807 08:51:28.268287 1212 ts_tablet_manager.cc:1397] T 05ec4889745e4ef5ac45cf5167e40533 P 3fff30b4948b409995102937d8eca802: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 08:51:28.270480 1214 raft_consensus.cc:348] T 05ec4889745e4ef5ac45cf5167e40533 P b0b7aa7bc88d4b08875d1f07d200ef0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0b7aa7bc88d4b08875d1f07d200ef0e" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 39809 } } peers { permanent_uuid: "98ad6bb1a58c4061ac1c40ade9b3c9ac" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 45753 } } peers { permanent_uuid: "3fff30b4948b409995102937d8eca802" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 39911 } }
I20260807 08:51:28.270635 1214 raft_consensus.cc:374] T 05ec4889745e4ef5ac45cf5167e40533 P b0b7aa7bc88d4b08875d1f07d200ef0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:51:28.270692 1214 raft_consensus.cc:729] T 05ec4889745e4ef5ac45cf5167e40533 P b0b7aa7bc88d4b08875d1f07d200ef0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0b7aa7bc88d4b08875d1f07d200ef0e, State: Initialized, Role: FOLLOWER
I20260807 08:51:28.270663 1212 raft_consensus.cc:348] T 05ec4889745e4ef5ac45cf5167e40533 P 3fff30b4948b409995102937d8eca802 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0b7aa7bc88d4b08875d1f07d200ef0e" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 39809 } } peers { permanent_uuid: "98ad6bb1a58c4061ac1c40ade9b3c9ac" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 45753 } } peers { permanent_uuid: "3fff30b4948b409995102937d8eca802" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 39911 } }
I20260807 08:51:28.270797 1212 raft_consensus.cc:374] T 05ec4889745e4ef5ac45cf5167e40533 P 3fff30b4948b409995102937d8eca802 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:51:28.270834 1212 raft_consensus.cc:729] T 05ec4889745e4ef5ac45cf5167e40533 P 3fff30b4948b409995102937d8eca802 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fff30b4948b409995102937d8eca802, State: Initialized, Role: FOLLOWER
I20260807 08:51:28.270803 1214 consensus_queue.cc:260] T 05ec4889745e4ef5ac45cf5167e40533 P b0b7aa7bc88d4b08875d1f07d200ef0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0b7aa7bc88d4b08875d1f07d200ef0e" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 39809 } } peers { permanent_uuid: "98ad6bb1a58c4061ac1c40ade9b3c9ac" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 45753 } } peers { permanent_uuid: "3fff30b4948b409995102937d8eca802" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 39911 } }
I20260807 08:51:28.270934 1212 consensus_queue.cc:260] T 05ec4889745e4ef5ac45cf5167e40533 P 3fff30b4948b409995102937d8eca802 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0b7aa7bc88d4b08875d1f07d200ef0e" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 39809 } } peers { permanent_uuid: "98ad6bb1a58c4061ac1c40ade9b3c9ac" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 45753 } } peers { permanent_uuid: "3fff30b4948b409995102937d8eca802" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 39911 } }
I20260807 08:51:28.271037 1214 ts_tablet_manager.cc:1428] T 05ec4889745e4ef5ac45cf5167e40533 P b0b7aa7bc88d4b08875d1f07d200ef0e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 08:51:28.271085 825 heartbeater.cc:503] Master 127.4.159.190:35953 was elected leader, sending a full tablet report...
I20260807 08:51:28.271167 1212 ts_tablet_manager.cc:1428] T 05ec4889745e4ef5ac45cf5167e40533 P 3fff30b4948b409995102937d8eca802: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 08:51:28.271271 1189 heartbeater.cc:503] Master 127.4.159.190:35953 was elected leader, sending a full tablet report...
W20260807 08:51:28.351709 1010 tablet.cc:2369] T 05ec4889745e4ef5ac45cf5167e40533 P 98ad6bb1a58c4061ac1c40ade9b3c9ac: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 08:51:28.443579 1227 raft_consensus.cc:482] T 05ec4889745e4ef5ac45cf5167e40533 P 3fff30b4948b409995102937d8eca802 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:51:28.443687 1227 raft_consensus.cc:504] T 05ec4889745e4ef5ac45cf5167e40533 P 3fff30b4948b409995102937d8eca802 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0b7aa7bc88d4b08875d1f07d200ef0e" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 39809 } } peers { permanent_uuid: "98ad6bb1a58c4061ac1c40ade9b3c9ac" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 45753 } } peers { permanent_uuid: "3fff30b4948b409995102937d8eca802" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 39911 } }
I20260807 08:51:28.443993 1227 leader_election.cc:288] T 05ec4889745e4ef5ac45cf5167e40533 P 3fff30b4948b409995102937d8eca802 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0b7aa7bc88d4b08875d1f07d200ef0e (127.4.159.129:39809), 98ad6bb1a58c4061ac1c40ade9b3c9ac (127.4.159.130:45753)
I20260807 08:51:28.446590 1196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:28.444136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.129:39809 (local address 127.4.159.131:56789)
0807 08:51:28.444218 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:28.444230 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 08:51:28.444283 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:51:28.444416 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:51:28.444418 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:51:28.444419 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:51:28.444475 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:28.444476 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:28.445058 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:28.445059 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:28.445811 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:28.445813 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:28.446371 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:28.446372 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:28.446430 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:28.446433 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 08:51:28.446477 (+ 44us) client_negotiation.cc:240] Negotiation successful
0807 08:51:28.446519 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62}
I20260807 08:51:28.446601 1207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:28.444252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.131:56789 (local address 127.4.159.129:39809)
0807 08:51:28.444301 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:28.444303 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:28.444312 (+ 9us) server_negotiation.cc:407] Connection header received
0807 08:51:28.444339 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:28.444339 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:28.444350 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:28.444402 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:51:28.444532 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:28.444993 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:28.445888 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:28.446281 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:28.446360 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:28.446369 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:28.446489 (+ 120us) server_negotiation.cc:299] Negotiation successful
0807 08:51:28.446545 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 08:51:28.446733 1206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:28.444189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.131:47335 (local address 127.4.159.130:45753)
0807 08:51:28.444233 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:28.444234 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:28.444389 (+ 155us) server_negotiation.cc:407] Connection header received
0807 08:51:28.444452 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:28.444453 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:28.444462 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:28.444504 (+ 42us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:51:28.444646 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:28.445084 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:28.445989 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:28.446388 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:28.446444 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:28.446452 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:28.446637 (+ 185us) server_negotiation.cc:299] Negotiation successful
0807 08:51:28.446671 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 08:51:28.446748 1228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:28.444136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.130:45753 (local address 127.4.159.131:47335)
0807 08:51:28.444331 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:28.444340 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 08:51:28.444396 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:51:28.444513 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:51:28.444514 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:51:28.444515 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:51:28.444586 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:28.444588 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:28.445155 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:28.445156 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:28.445904 (+ 748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:28.445906 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:28.446464 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:28.446466 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:28.446544 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:28.446548 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 08:51:28.446623 (+ 75us) client_negotiation.cc:240] Negotiation successful
0807 08:51:28.446660 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"mutex_wait_us":32,"thread_start_us":89,"threads_started":1}
I20260807 08:51:28.446938 759 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05ec4889745e4ef5ac45cf5167e40533" candidate_uuid: "3fff30b4948b409995102937d8eca802" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0b7aa7bc88d4b08875d1f07d200ef0e" is_pre_election: true
I20260807 08:51:28.447024 943 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05ec4889745e4ef5ac45cf5167e40533" candidate_uuid: "3fff30b4948b409995102937d8eca802" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98ad6bb1a58c4061ac1c40ade9b3c9ac" is_pre_election: true
I20260807 08:51:28.447115 759 raft_consensus.cc:2454] T 05ec4889745e4ef5ac45cf5167e40533 P b0b7aa7bc88d4b08875d1f07d200ef0e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3fff30b4948b409995102937d8eca802 in term 0.
I20260807 08:51:28.447146 943 raft_consensus.cc:2454] T 05ec4889745e4ef5ac45cf5167e40533 P 98ad6bb1a58c4061ac1c40ade9b3c9ac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3fff30b4948b409995102937d8eca802 in term 0.
I20260807 08:51:28.447367 1039 leader_election.cc:302] T 05ec4889745e4ef5ac45cf5167e40533 P 3fff30b4948b409995102937d8eca802 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fff30b4948b409995102937d8eca802, 98ad6bb1a58c4061ac1c40ade9b3c9ac; no voters:
I20260807 08:51:28.447505 1227 raft_consensus.cc:2781] T 05ec4889745e4ef5ac45cf5167e40533 P 3fff30b4948b409995102937d8eca802 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:51:28.447561 1227 raft_consensus.cc:482] T 05ec4889745e4ef5ac45cf5167e40533 P 3fff30b4948b409995102937d8eca802 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:51:28.447582 1227 raft_consensus.cc:3037] T 05ec4889745e4ef5ac45cf5167e40533 P 3fff30b4948b409995102937d8eca802 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:51:28.448210 1227 raft_consensus.cc:504] T 05ec4889745e4ef5ac45cf5167e40533 P 3fff30b4948b409995102937d8eca802 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0b7aa7bc88d4b08875d1f07d200ef0e" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 39809 } } peers { permanent_uuid: "98ad6bb1a58c4061ac1c40ade9b3c9ac" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 45753 } } peers { permanent_uuid: "3fff30b4948b409995102937d8eca802" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 39911 } }
I20260807 08:51:28.448328 1227 leader_election.cc:288] T 05ec4889745e4ef5ac45cf5167e40533 P 3fff30b4948b409995102937d8eca802 [CANDIDATE]: Term 1 election: Requested vote from peers b0b7aa7bc88d4b08875d1f07d200ef0e (127.4.159.129:39809), 98ad6bb1a58c4061ac1c40ade9b3c9ac (127.4.159.130:45753)
I20260807 08:51:28.448473 759 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05ec4889745e4ef5ac45cf5167e40533" candidate_uuid: "3fff30b4948b409995102937d8eca802" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0b7aa7bc88d4b08875d1f07d200ef0e"
I20260807 08:51:28.448544 943 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05ec4889745e4ef5ac45cf5167e40533" candidate_uuid: "3fff30b4948b409995102937d8eca802" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98ad6bb1a58c4061ac1c40ade9b3c9ac"
I20260807 08:51:28.448565 759 raft_consensus.cc:3037] T 05ec4889745e4ef5ac45cf5167e40533 P b0b7aa7bc88d4b08875d1f07d200ef0e [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:51:28.448617 943 raft_consensus.cc:3037] T 05ec4889745e4ef5ac45cf5167e40533 P 98ad6bb1a58c4061ac1c40ade9b3c9ac [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:51:28.449448 759 raft_consensus.cc:2454] T 05ec4889745e4ef5ac45cf5167e40533 P b0b7aa7bc88d4b08875d1f07d200ef0e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fff30b4948b409995102937d8eca802 in term 1.
I20260807 08:51:28.449448 943 raft_consensus.cc:2454] T 05ec4889745e4ef5ac45cf5167e40533 P 98ad6bb1a58c4061ac1c40ade9b3c9ac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fff30b4948b409995102937d8eca802 in term 1.
I20260807 08:51:28.449592 1039 leader_election.cc:302] T 05ec4889745e4ef5ac45cf5167e40533 P 3fff30b4948b409995102937d8eca802 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fff30b4948b409995102937d8eca802, 98ad6bb1a58c4061ac1c40ade9b3c9ac; no voters:
I20260807 08:51:28.449683 1227 raft_consensus.cc:2781] T 05ec4889745e4ef5ac45cf5167e40533 P 3fff30b4948b409995102937d8eca802 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:51:28.449802 1227 raft_consensus.cc:686] T 05ec4889745e4ef5ac45cf5167e40533 P 3fff30b4948b409995102937d8eca802 [term 1 LEADER]: Becoming Leader. State: Replica: 3fff30b4948b409995102937d8eca802, State: Running, Role: LEADER
I20260807 08:51:28.449882 1227 consensus_queue.cc:237] T 05ec4889745e4ef5ac45cf5167e40533 P 3fff30b4948b409995102937d8eca802 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0b7aa7bc88d4b08875d1f07d200ef0e" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 39809 } } peers { permanent_uuid: "98ad6bb1a58c4061ac1c40ade9b3c9ac" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 45753 } } peers { permanent_uuid: "3fff30b4948b409995102937d8eca802" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 39911 } }
I20260807 08:51:28.450592 541 catalog_manager.cc:5495] T 05ec4889745e4ef5ac45cf5167e40533 P 3fff30b4948b409995102937d8eca802 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3fff30b4948b409995102937d8eca802 (127.4.159.131). New cstate: current_term: 1 leader_uuid: "3fff30b4948b409995102937d8eca802" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0b7aa7bc88d4b08875d1f07d200ef0e" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 39809 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "98ad6bb1a58c4061ac1c40ade9b3c9ac" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 45753 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3fff30b4948b409995102937d8eca802" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 39911 } health_report { overall_health: HEALTHY } } }
I20260807 08:51:28.480535 1209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:28.478647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39628 (local address 127.4.159.131:39911)
0807 08:51:28.478703 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:28.478704 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:28.478726 (+ 22us) server_negotiation.cc:407] Connection header received
0807 08:51:28.478755 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:28.478756 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:28.478771 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:28.478823 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:51:28.478902 (+ 79us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:28.479328 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:28.479955 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:28.480116 (+ 161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:28.480155 (+ 39us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:28.480277 (+ 122us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:51:28.480353 (+ 76us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:51:28.480415 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:28.480448 (+ 33us) server_negotiation.cc:299] Negotiation successful
0807 08:51:28.480455 (+ 7us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260807 08:51:28.482029 759 raft_consensus.cc:1261] T 05ec4889745e4ef5ac45cf5167e40533 P b0b7aa7bc88d4b08875d1f07d200ef0e [term 1 FOLLOWER]: Refusing update from remote peer 3fff30b4948b409995102937d8eca802: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:51:28.482038 943 raft_consensus.cc:1261] T 05ec4889745e4ef5ac45cf5167e40533 P 98ad6bb1a58c4061ac1c40ade9b3c9ac [term 1 FOLLOWER]: Refusing update from remote peer 3fff30b4948b409995102937d8eca802: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260807 08:51:28.482100 1194 tablet.cc:2369] T 05ec4889745e4ef5ac45cf5167e40533 P 3fff30b4948b409995102937d8eca802: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 08:51:28.482363 1227 consensus_queue.cc:1035] T 05ec4889745e4ef5ac45cf5167e40533 P 3fff30b4948b409995102937d8eca802 [LEADER]: Connected to new peer: Peer: permanent_uuid: "98ad6bb1a58c4061ac1c40ade9b3c9ac" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 45753 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:51:28.482492 1229 consensus_queue.cc:1035] T 05ec4889745e4ef5ac45cf5167e40533 P 3fff30b4948b409995102937d8eca802 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0b7aa7bc88d4b08875d1f07d200ef0e" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 39809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 08:51:28.484203 826 tablet.cc:2369] T 05ec4889745e4ef5ac45cf5167e40533 P b0b7aa7bc88d4b08875d1f07d200ef0e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 08:51:28.485526 1239 mvcc.cc:205] Tried to move back new op lower bound from 7315835652020219904 to 7315835651891302400. Current Snapshot: MvccSnapshot[applied={T|T < 7315835652020219904 or (T in {7315835652020219904})}]
I20260807 08:51:28.486362 1236 mvcc.cc:205] Tried to move back new op lower bound from 7315835652020219904 to 7315835651891302400. Current Snapshot: MvccSnapshot[applied={T|T < 7315835652020219904 or (T in {7315835652020219904})}]
I20260807 08:51:28.486610 538 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49998:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:51:28.486706 538 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49998:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:51:28.487378 538 catalog_manager.cc:5782] T 00000000000000000000000000000000 P eab89cccc4684dffa92ac33f8dbb9334: Sending DeleteTablet for 3 replicas of tablet 05ec4889745e4ef5ac45cf5167e40533
I20260807 08:51:28.487666 915 tablet_service.cc:1514] Processing DeleteTablet for tablet 05ec4889745e4ef5ac45cf5167e40533 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:51:28 UTC) from {username='slave'} at 127.0.0.1:42002
I20260807 08:51:28.487725 729 tablet_service.cc:1514] Processing DeleteTablet for tablet 05ec4889745e4ef5ac45cf5167e40533 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:51:28 UTC) from {username='slave'} at 127.0.0.1:55242
I20260807 08:51:28.487948 1252 tablet_replica.cc:330] T 05ec4889745e4ef5ac45cf5167e40533 P b0b7aa7bc88d4b08875d1f07d200ef0e: stopping tablet replica
I20260807 08:51:28.487987 1097 tablet_service.cc:1514] Processing DeleteTablet for tablet 05ec4889745e4ef5ac45cf5167e40533 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:51:28 UTC) from {username='slave'} at 127.0.0.1:39616
I20260807 08:51:28.488096 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 630
I20260807 08:51:28.488139 1252 raft_consensus.cc:2229] T 05ec4889745e4ef5ac45cf5167e40533 P b0b7aa7bc88d4b08875d1f07d200ef0e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:51:28.488198 1251 tablet_replica.cc:330] T 05ec4889745e4ef5ac45cf5167e40533 P 98ad6bb1a58c4061ac1c40ade9b3c9ac: stopping tablet replica
I20260807 08:51:28.488358 1252 raft_consensus.cc:2258] T 05ec4889745e4ef5ac45cf5167e40533 P b0b7aa7bc88d4b08875d1f07d200ef0e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:51:28.488632 1251 raft_consensus.cc:2229] T 05ec4889745e4ef5ac45cf5167e40533 P 98ad6bb1a58c4061ac1c40ade9b3c9ac [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:51:28.488761 1251 raft_consensus.cc:2258] T 05ec4889745e4ef5ac45cf5167e40533 P 98ad6bb1a58c4061ac1c40ade9b3c9ac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:51:28.488914 1254 tablet_replica.cc:330] T 05ec4889745e4ef5ac45cf5167e40533 P 3fff30b4948b409995102937d8eca802: stopping tablet replica
I20260807 08:51:28.488986 1254 raft_consensus.cc:2229] T 05ec4889745e4ef5ac45cf5167e40533 P 3fff30b4948b409995102937d8eca802 [term 1 LEADER]: Raft consensus shutting down.
I20260807 08:51:28.489048 1254 raft_consensus.cc:2258] T 05ec4889745e4ef5ac45cf5167e40533 P 3fff30b4948b409995102937d8eca802 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:51:28.492183 1251 ts_tablet_manager.cc:1905] T 05ec4889745e4ef5ac45cf5167e40533 P 98ad6bb1a58c4061ac1c40ade9b3c9ac: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:51:28.493774 1251 ts_tablet_manager.cc:1918] T 05ec4889745e4ef5ac45cf5167e40533 P 98ad6bb1a58c4061ac1c40ade9b3c9ac: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:51:28.493834 1251 log.cc:1196] T 05ec4889745e4ef5ac45cf5167e40533 P 98ad6bb1a58c4061ac1c40ade9b3c9ac: Deleting WAL directory at /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786092453486433-4734-0/minicluster-data/ts-1/wal/wals/05ec4889745e4ef5ac45cf5167e40533
I20260807 08:51:28.494045 1251 ts_tablet_manager.cc:1939] T 05ec4889745e4ef5ac45cf5167e40533 P 98ad6bb1a58c4061ac1c40ade9b3c9ac: Deleting consensus metadata
I20260807 08:51:28.494427 517 catalog_manager.cc:4841] TS 98ad6bb1a58c4061ac1c40ade9b3c9ac (127.4.159.130:45753): tablet 05ec4889745e4ef5ac45cf5167e40533 (table test-table [id=338a72be143641628531d4731e588d55]) successfully deleted
I20260807 08:51:28.494544 1254 ts_tablet_manager.cc:1905] T 05ec4889745e4ef5ac45cf5167e40533 P 3fff30b4948b409995102937d8eca802: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:51:28.495620 1254 ts_tablet_manager.cc:1918] T 05ec4889745e4ef5ac45cf5167e40533 P 3fff30b4948b409995102937d8eca802: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:51:28.495680 1254 log.cc:1196] T 05ec4889745e4ef5ac45cf5167e40533 P 3fff30b4948b409995102937d8eca802: Deleting WAL directory at /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786092453486433-4734-0/minicluster-data/ts-2/wal/wals/05ec4889745e4ef5ac45cf5167e40533
I20260807 08:51:28.495885 1254 ts_tablet_manager.cc:1939] T 05ec4889745e4ef5ac45cf5167e40533 P 3fff30b4948b409995102937d8eca802: Deleting consensus metadata
I20260807 08:51:28.496198 519 catalog_manager.cc:4841] TS 3fff30b4948b409995102937d8eca802 (127.4.159.131:39911): tablet 05ec4889745e4ef5ac45cf5167e40533 (table test-table [id=338a72be143641628531d4731e588d55]) successfully deleted
W20260807 08:51:28.503290 519 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.159.129:39809 (error 108)
I20260807 08:51:28.503510 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 829
I20260807 08:51:28.503726 1205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:28.503624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.129:39809 (local address 127.0.0.1:55246)
0807 08:51:28.503665 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:28.503696 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.129:39809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260807 08:51:28.503819 519 catalog_manager.cc:4584] TS b0b7aa7bc88d4b08875d1f07d200ef0e (127.4.159.129:39809): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 05ec4889745e4ef5ac45cf5167e40533: Network error: Client connection negotiation failed: client connection to 127.4.159.129:39809: connect: Connection refused (error 111)
I20260807 08:51:28.510955 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 1013
I20260807 08:51:28.521823 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 487
2026-08-07T08:51:28Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (845 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786092453486433-4734-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:51:28 dist-test-slave-6zbq krb5kdc[1258](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:51:28 dist-test-slave-6zbq krb5kdc[1258](info): set up 2 sockets
Aug 07 08:51:28 dist-test-slave-6zbq krb5kdc[1258](info): commencing operation
krb5kdc: starting...
W20260807 08:51:29.645191 4734 mini_kdc.cc:121] Time spent starting KDC: real 1.101s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786092453486433-4734-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786092453486433-4734-0/krb5kdc/test-user.keytab.
Aug 07 08:51:29 dist-test-slave-6zbq krb5kdc[1258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092689, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:51:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:51:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.159.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.159.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.159.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab.
Entry for principal kudu/127.4.159.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab.
I20260807 08:51:29.753682 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786092453486433-4734-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786092453486433-4734-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786092453486433-4734-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.159.190:46243
--webserver_interface=127.4.159.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab
--principal=kudu/127.4.159.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:44215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.159.190:46243
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:51:29.832794 1278 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:51:29.832974 1278 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:51:29.833003 1278 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:51:29.834077 1278 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:51:29.834115 1278 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:51:29.834133 1278 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:51:29.834148 1278 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:51:29.834165 1278 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:51:29.835619 1278 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:44215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786092453486433-4734-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786092453486433-4734-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.159.190:46243
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab
--principal=kudu/127.4.159.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.159.190:46243
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.159.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1278
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786092453486433-4734-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:51:29.835850 1278 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:51:29.836040 1278 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:51:29.840440 1285 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:51:29.840430 1284 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:51:29.840440 1287 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:51:29.840459 1278 server_base.cc:1034] running on GCE node
I20260807 08:51:29.840879 1278 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:51:29.841153 1278 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:51:29.842293 1278 hybrid_clock.cc:648] HybridClock initialized: now 1786092689842276 us; error 27 us; skew 500 ppm
Aug 07 08:51:29 dist-test-slave-6zbq krb5kdc[1258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092689, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:51:29.844560 1278 init.cc:377] Logged in from keytab as kudu/127.4.159.190@KRBTEST.COM (short username kudu)
I20260807 08:51:29.844933 1278 webserver.cc:460] Webserver started at http://127.4.159.190:39433/ using document root <none> and password file <none>
I20260807 08:51:29.845109 1278 fs_manager.cc:359] Metadata directory not provided
I20260807 08:51:29.845153 1278 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:51:29.845279 1278 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:51:29.846215 1278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786092453486433-4734-0/minicluster-data/master-0/data/instance:
uuid: "b59b19d11e64484f8c4a45f30423dc68"
format_stamp: "Formatted at 2026-08-07 08:51:29 on dist-test-slave-6zbq"
server_key: "23450e73480c7ba920bb4f3d3f4c9029"
server_key_iv: "b6500cb2c3f2bdcc116f62ef60e879db"
server_key_version: "encryptionkey@0"
I20260807 08:51:29.846580 1278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786092453486433-4734-0/minicluster-data/master-0/wal/instance:
uuid: "b59b19d11e64484f8c4a45f30423dc68"
format_stamp: "Formatted at 2026-08-07 08:51:29 on dist-test-slave-6zbq"
server_key: "23450e73480c7ba920bb4f3d3f4c9029"
server_key_iv: "b6500cb2c3f2bdcc116f62ef60e879db"
server_key_version: "encryptionkey@0"
I20260807 08:51:29.847865 1278 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:51:29.848488 1297 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:51:29.848631 1278 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 08:51:29.848709 1278 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786092453486433-4734-0/minicluster-data/master-0/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786092453486433-4734-0/minicluster-data/master-0/wal
uuid: "b59b19d11e64484f8c4a45f30423dc68"
format_stamp: "Formatted at 2026-08-07 08:51:29 on dist-test-slave-6zbq"
server_key: "23450e73480c7ba920bb4f3d3f4c9029"
server_key_iv: "b6500cb2c3f2bdcc116f62ef60e879db"
server_key_version: "encryptionkey@0"
I20260807 08:51:29.848773 1278 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786092453486433-4734-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786092453486433-4734-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786092453486433-4734-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:51:29.868645 1278 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:51:29.870112 1278 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:51:29.870234 1278 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:51:29.873730 1278 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.190:46243
I20260807 08:51:29.873780 1376 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.190:46243 every 8 connection(s)
I20260807 08:51:29.874045 1278 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
I20260807 08:51:29.874816 1378 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:51:29.876843 1378 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b59b19d11e64484f8c4a45f30423dc68: Bootstrap starting.
I20260807 08:51:29.877408 1378 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b59b19d11e64484f8c4a45f30423dc68: Neither blocks nor log segments found. Creating new log.
I20260807 08:51:29.877417 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 1278
I20260807 08:51:29.877517 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786092453486433-4734-0/minicluster-data/master-0/wal/instance
I20260807 08:51:29.877707 4734 external_mini_cluster.cc:1428] Setting key 096f2459622651830a9165171566ba03
I20260807 08:51:29.877719 1378 log.cc:824] T 00000000000000000000000000000000 P b59b19d11e64484f8c4a45f30423dc68: Log is configured to *not* fsync() on all Append() calls
I20260807 08:51:29.878473 1378 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b59b19d11e64484f8c4a45f30423dc68: No bootstrap required, opened a new log
I20260807 08:51:29.880314 1378 raft_consensus.cc:348] T 00000000000000000000000000000000 P b59b19d11e64484f8c4a45f30423dc68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b59b19d11e64484f8c4a45f30423dc68" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 46243 } }
I20260807 08:51:29.880435 1378 raft_consensus.cc:374] T 00000000000000000000000000000000 P b59b19d11e64484f8c4a45f30423dc68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:51:29.880455 1378 raft_consensus.cc:729] T 00000000000000000000000000000000 P b59b19d11e64484f8c4a45f30423dc68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b59b19d11e64484f8c4a45f30423dc68, State: Initialized, Role: FOLLOWER
I20260807 08:51:29.880555 1378 consensus_queue.cc:260] T 00000000000000000000000000000000 P b59b19d11e64484f8c4a45f30423dc68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b59b19d11e64484f8c4a45f30423dc68" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 46243 } }
I20260807 08:51:29.880625 1378 raft_consensus.cc:388] T 00000000000000000000000000000000 P b59b19d11e64484f8c4a45f30423dc68 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:51:29.880654 1378 raft_consensus.cc:482] T 00000000000000000000000000000000 P b59b19d11e64484f8c4a45f30423dc68 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:51:29.880699 1378 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b59b19d11e64484f8c4a45f30423dc68 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:51:29.881191 1378 raft_consensus.cc:504] T 00000000000000000000000000000000 P b59b19d11e64484f8c4a45f30423dc68 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b59b19d11e64484f8c4a45f30423dc68" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 46243 } }
I20260807 08:51:29.881285 1378 leader_election.cc:302] T 00000000000000000000000000000000 P b59b19d11e64484f8c4a45f30423dc68 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b59b19d11e64484f8c4a45f30423dc68; no voters:
I20260807 08:51:29.881424 1378 leader_election.cc:288] T 00000000000000000000000000000000 P b59b19d11e64484f8c4a45f30423dc68 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:51:29.881471 1388 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b59b19d11e64484f8c4a45f30423dc68 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:51:29.881604 1388 raft_consensus.cc:686] T 00000000000000000000000000000000 P b59b19d11e64484f8c4a45f30423dc68 [term 1 LEADER]: Becoming Leader. State: Replica: b59b19d11e64484f8c4a45f30423dc68, State: Running, Role: LEADER
I20260807 08:51:29.881641 1378 sys_catalog.cc:564] T 00000000000000000000000000000000 P b59b19d11e64484f8c4a45f30423dc68 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:51:29.881686 1388 consensus_queue.cc:237] T 00000000000000000000000000000000 P b59b19d11e64484f8c4a45f30423dc68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b59b19d11e64484f8c4a45f30423dc68" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 46243 } }
I20260807 08:51:29.882033 1390 sys_catalog.cc:455] T 00000000000000000000000000000000 P b59b19d11e64484f8c4a45f30423dc68 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b59b19d11e64484f8c4a45f30423dc68" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b59b19d11e64484f8c4a45f30423dc68" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 46243 } } }
I20260807 08:51:29.882113 1390 sys_catalog.cc:458] T 00000000000000000000000000000000 P b59b19d11e64484f8c4a45f30423dc68 [sys.catalog]: This master's current role is: LEADER
I20260807 08:51:29.882623 1393 sys_catalog.cc:455] T 00000000000000000000000000000000 P b59b19d11e64484f8c4a45f30423dc68 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b59b19d11e64484f8c4a45f30423dc68. Latest consensus state: current_term: 1 leader_uuid: "b59b19d11e64484f8c4a45f30423dc68" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b59b19d11e64484f8c4a45f30423dc68" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 46243 } } }
I20260807 08:51:29.882701 1393 sys_catalog.cc:458] T 00000000000000000000000000000000 P b59b19d11e64484f8c4a45f30423dc68 [sys.catalog]: This master's current role is: LEADER
I20260807 08:51:29.882815 1408 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:51:29.883239 1408 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 07 08:51:29 dist-test-slave-6zbq krb5kdc[1258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092689, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.159.190@KRBTEST.COM
I20260807 08:51:29.884930 1408 catalog_manager.cc:1347] Generated new cluster ID: cd6e12a633634f8a9913c5885f1009f0
I20260807 08:51:29.884979 1408 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:51:29.888299 1386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:29.878136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43074 (local address 127.4.159.190:46243)
0807 08:51:29.878306 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:29.878315 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:29.878362 (+ 47us) server_negotiation.cc:407] Connection header received
0807 08:51:29.878798 (+ 436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:29.878806 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:29.878852 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:29.878966 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:51:29.879559 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:29.880390 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:29.881213 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:29.881801 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:29.882190 (+ 389us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:29.884778 (+ 2588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:51:29.884782 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:51:29.884783 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:51:29.884794 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:51:29.886779 (+ 1985us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:51:29.887138 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:51:29.887140 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:51:29.887141 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:51:29.887254 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:51:29.887431 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:51:29.887432 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:51:29.887432 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:51:29.887725 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:51:29.887882 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:29.887957 (+ 75us) server_negotiation.cc:299] Negotiation successful
0807 08:51:29.888088 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":78,"threads_started":1}
I20260807 08:51:29.899665 1408 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:51:29.900112 1408 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:51:29.902380 1408 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b59b19d11e64484f8c4a45f30423dc68: Generated new TSK 0
I20260807 08:51:29.902515 1408 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.159.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.159.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.159.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.129.keytab.
Entry for principal kudu/127.4.159.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.129.keytab.
I20260807 08:51:29.926008 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786092453486433-4734-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786092453486433-4734-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786092453486433-4734-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786092453486433-4734-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.159.129:0
--local_ip_for_outbound_sockets=127.4.159.129
--webserver_interface=127.4.159.129
--webserver_port=0
--tserver_master_addrs=127.4.159.190:46243
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.129.keytab
--principal=kudu/127.4.159.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:44215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:51:30.004194 1425 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:51:30.004386 1425 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:51:30.004418 1425 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:51:30.005527 1425 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:51:30.005564 1425 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:51:30.005599 1425 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.159.129
I20260807 08:51:30.007023 1425 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:44215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786092453486433-4734-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786092453486433-4734-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.129.keytab
--principal=kudu/127.4.159.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.159.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786092453486433-4734-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.159.129
--webserver_port=0
--tserver_master_addrs=127.4.159.190:46243
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1425
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.159.129
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786092453486433-4734-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:51:30.007256 1425 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:51:30.007467 1425 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:51:30.011906 1437 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:51:30.011920 1440 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:51:30.011977 1425 server_base.cc:1034] running on GCE node
W20260807 08:51:30.011922 1434 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:51:30.012365 1425 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:51:30.012547 1425 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:51:30.013671 1425 hybrid_clock.cc:648] HybridClock initialized: now 1786092690013661 us; error 29 us; skew 500 ppm
Aug 07 08:51:30 dist-test-slave-6zbq krb5kdc[1258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092690, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:51:30.015910 1425 init.cc:377] Logged in from keytab as kudu/127.4.159.129@KRBTEST.COM (short username kudu)
I20260807 08:51:30.016304 1425 webserver.cc:460] Webserver started at http://127.4.159.129:34189/ using document root <none> and password file <none>
I20260807 08:51:30.016475 1425 fs_manager.cc:359] Metadata directory not provided
I20260807 08:51:30.016511 1425 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:51:30.016598 1425 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:51:30.017503 1425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786092453486433-4734-0/minicluster-data/ts-0/data/instance:
uuid: "25d93c2f00e544eba4c891390e740d56"
format_stamp: "Formatted at 2026-08-07 08:51:30 on dist-test-slave-6zbq"
server_key: "55aa965398df67cdaeca3b3968eee83f"
server_key_iv: "1839737cac860e5d9f1cc519dc690ad8"
server_key_version: "encryptionkey@0"
I20260807 08:51:30.017772 1425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786092453486433-4734-0/minicluster-data/ts-0/wal/instance:
uuid: "25d93c2f00e544eba4c891390e740d56"
format_stamp: "Formatted at 2026-08-07 08:51:30 on dist-test-slave-6zbq"
server_key: "55aa965398df67cdaeca3b3968eee83f"
server_key_iv: "1839737cac860e5d9f1cc519dc690ad8"
server_key_version: "encryptionkey@0"
I20260807 08:51:30.018918 1425 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 08:51:30.019506 1451 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:51:30.019654 1425 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 08:51:30.019729 1425 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786092453486433-4734-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786092453486433-4734-0/minicluster-data/ts-0/wal
uuid: "25d93c2f00e544eba4c891390e740d56"
format_stamp: "Formatted at 2026-08-07 08:51:30 on dist-test-slave-6zbq"
server_key: "55aa965398df67cdaeca3b3968eee83f"
server_key_iv: "1839737cac860e5d9f1cc519dc690ad8"
server_key_version: "encryptionkey@0"
I20260807 08:51:30.019781 1425 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786092453486433-4734-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786092453486433-4734-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786092453486433-4734-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:51:30.040087 1425 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:51:30.041754 1425 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:51:30.041850 1425 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:51:30.042001 1425 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:51:30.042227 1425 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:51:30.042256 1425 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:51:30.042274 1425 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:51:30.042289 1425 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:51:30.047335 1425 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.129:34869
I20260807 08:51:30.047394 1621 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.129:34869 every 8 connection(s)
I20260807 08:51:30.047633 1425 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786092453486433-4734-0/minicluster-data/ts-0/data/info.pb
I20260807 08:51:30.050516 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 1425
I20260807 08:51:30.050625 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786092453486433-4734-0/minicluster-data/ts-0/wal/instance
I20260807 08:51:30.050810 4734 external_mini_cluster.cc:1428] Setting key 7f80bc79b2f54de784e0111342c4c215
Aug 07 08:51:30 dist-test-slave-6zbq krb5kdc[1258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092690, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.129@KRBTEST.COM for kudu/127.4.159.190@KRBTEST.COM
I20260807 08:51:30.056229 1386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:30.048231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.129:58055 (local address 127.4.159.190:46243)
0807 08:51:30.048283 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:30.048285 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:30.048448 (+ 163us) server_negotiation.cc:407] Connection header received
0807 08:51:30.048851 (+ 403us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:30.048852 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:30.048860 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:30.048909 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:51:30.049708 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:30.050197 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:30.051434 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:30.051664 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:30.051712 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:30.053720 (+ 2008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:51:30.053722 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:51:30.053722 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:51:30.053729 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:51:30.055000 (+ 1271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:51:30.055364 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:51:30.055365 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:51:30.055365 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:51:30.055421 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:51:30.055633 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:51:30.055633 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:51:30.055634 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:51:30.055744 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:51:30.055799 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:30.056040 (+ 241us) server_negotiation.cc:299] Negotiation successful
0807 08:51:30.056135 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260807 08:51:30.056357 1624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:30.048198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:46243 (local address 127.4.159.129:58055)
0807 08:51:30.048340 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:30.048377 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 08:51:30.048782 (+ 405us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:51:30.048955 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:51:30.048958 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:51:30.049466 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:51:30.049641 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:30.049645 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:30.050263 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:30.050264 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:30.051346 (+ 1082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:30.051349 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:30.051740 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:30.051741 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:30.051866 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:30.052259 (+ 393us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:51:30.052268 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:51:30.053634 (+ 1366us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:51:30.055075 (+ 1441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:51:30.055082 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:51:30.055084 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:51:30.055297 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:51:30.055475 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:51:30.055476 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:51:30.055476 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:51:30.055577 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:51:30.055800 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:51:30.055806 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:51:30.055929 (+ 123us) client_negotiation.cc:769] Sending connection context
0807 08:51:30.056011 (+ 82us) client_negotiation.cc:240] Negotiation successful
0807 08:51:30.056151 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":67,"threads_started":1}
I20260807 08:51:30.056770 1622 heartbeater.cc:347] Connected to a master server at 127.4.159.190:46243
I20260807 08:51:30.056851 1622 heartbeater.cc:464] Registering TS with master...
I20260807 08:51:30.057008 1622 heartbeater.cc:511] Master 127.4.159.190:46243 requested a full tablet report, sending...
I20260807 08:51:30.057474 1323 ts_manager.cc:194] Registered new tserver with Master: 25d93c2f00e544eba4c891390e740d56 (127.4.159.129:34869)
I20260807 08:51:30.058032 1323 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.159.129@KRBTEST.COM'} at 127.4.159.129:58055
WARNING: no policy specified for kudu/127.4.159.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.159.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.159.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.130.keytab.
Entry for principal kudu/127.4.159.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.130.keytab.
I20260807 08:51:30.072456 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786092453486433-4734-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786092453486433-4734-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786092453486433-4734-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786092453486433-4734-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.159.130:0
--local_ip_for_outbound_sockets=127.4.159.130
--webserver_interface=127.4.159.130
--webserver_port=0
--tserver_master_addrs=127.4.159.190:46243
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.130.keytab
--principal=kudu/127.4.159.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:44215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:51:30.150108 1627 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:51:30.150280 1627 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:51:30.150301 1627 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:51:30.151476 1627 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:51:30.151513 1627 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:51:30.151542 1627 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.159.130
I20260807 08:51:30.152961 1627 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:44215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786092453486433-4734-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786092453486433-4734-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.130.keytab
--principal=kudu/127.4.159.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.159.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786092453486433-4734-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.159.130
--webserver_port=0
--tserver_master_addrs=127.4.159.190:46243
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1627
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.159.130
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786092453486433-4734-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:51:30.153155 1627 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:51:30.153340 1627 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:51:30.157809 1634 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:51:30.157815 1627 server_base.cc:1034] running on GCE node
W20260807 08:51:30.157788 1636 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:51:30.157779 1638 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:51:30.158144 1627 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:51:30.158403 1627 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:51:30.159541 1627 hybrid_clock.cc:648] HybridClock initialized: now 1786092690159526 us; error 26 us; skew 500 ppm
Aug 07 08:51:30 dist-test-slave-6zbq krb5kdc[1258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092690, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:51:30.161697 1627 init.cc:377] Logged in from keytab as kudu/127.4.159.130@KRBTEST.COM (short username kudu)
I20260807 08:51:30.162065 1627 webserver.cc:460] Webserver started at http://127.4.159.130:43639/ using document root <none> and password file <none>
I20260807 08:51:30.162271 1627 fs_manager.cc:359] Metadata directory not provided
I20260807 08:51:30.162317 1627 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:51:30.162458 1627 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:51:30.163374 1627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786092453486433-4734-0/minicluster-data/ts-1/data/instance:
uuid: "bb79bb3fdbb94fe2b86bd969b9642d62"
format_stamp: "Formatted at 2026-08-07 08:51:30 on dist-test-slave-6zbq"
server_key: "bf41c9c96b720ad6d49f1695e0de87cb"
server_key_iv: "b529ec4ba5afac6cb74d9b4a58914578"
server_key_version: "encryptionkey@0"
I20260807 08:51:30.163681 1627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786092453486433-4734-0/minicluster-data/ts-1/wal/instance:
uuid: "bb79bb3fdbb94fe2b86bd969b9642d62"
format_stamp: "Formatted at 2026-08-07 08:51:30 on dist-test-slave-6zbq"
server_key: "bf41c9c96b720ad6d49f1695e0de87cb"
server_key_iv: "b529ec4ba5afac6cb74d9b4a58914578"
server_key_version: "encryptionkey@0"
I20260807 08:51:30.164882 1627 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 08:51:30.165480 1652 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:51:30.165622 1627 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 08:51:30.165690 1627 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786092453486433-4734-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786092453486433-4734-0/minicluster-data/ts-1/wal
uuid: "bb79bb3fdbb94fe2b86bd969b9642d62"
format_stamp: "Formatted at 2026-08-07 08:51:30 on dist-test-slave-6zbq"
server_key: "bf41c9c96b720ad6d49f1695e0de87cb"
server_key_iv: "b529ec4ba5afac6cb74d9b4a58914578"
server_key_version: "encryptionkey@0"
I20260807 08:51:30.165756 1627 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786092453486433-4734-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786092453486433-4734-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786092453486433-4734-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:51:30.183472 1627 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:51:30.184957 1627 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:51:30.185050 1627 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:51:30.185226 1627 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:51:30.185449 1627 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:51:30.185479 1627 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:51:30.185496 1627 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:51:30.185508 1627 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:51:30.190657 1627 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.130:38859
I20260807 08:51:30.190721 1820 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.130:38859 every 8 connection(s)
I20260807 08:51:30.190979 1627 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786092453486433-4734-0/minicluster-data/ts-1/data/info.pb
I20260807 08:51:30.196305 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 1627
I20260807 08:51:30.196413 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786092453486433-4734-0/minicluster-data/ts-1/wal/instance
Aug 07 08:51:30 dist-test-slave-6zbq krb5kdc[1258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092690, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.130@KRBTEST.COM for kudu/127.4.159.190@KRBTEST.COM
I20260807 08:51:30.196586 4734 external_mini_cluster.cc:1428] Setting key 956be3e3415820fcfeb53cbfcaf4ade1
I20260807 08:51:30.199515 1386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:30.191594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.130:50625 (local address 127.4.159.190:46243)
0807 08:51:30.191651 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:30.191653 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:30.191835 (+ 182us) server_negotiation.cc:407] Connection header received
0807 08:51:30.192369 (+ 534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:30.192370 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:30.192383 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:30.192433 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:51:30.193283 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:30.193772 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:30.194720 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:30.194919 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:30.194974 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:30.196973 (+ 1999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:51:30.196974 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:51:30.196974 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:51:30.196982 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:51:30.198284 (+ 1302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:51:30.198661 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:51:30.198662 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:51:30.198662 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:51:30.198724 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:51:30.198934 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:51:30.198936 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:51:30.198936 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:51:30.199051 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:51:30.199109 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:30.199346 (+ 237us) server_negotiation.cc:299] Negotiation successful
0807 08:51:30.199426 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260807 08:51:30.199656 1823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:30.191584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:46243 (local address 127.4.159.130:50625)
0807 08:51:30.191719 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:30.191759 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 08:51:30.192267 (+ 508us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:51:30.192493 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:51:30.192497 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:51:30.193010 (+ 513us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:51:30.193183 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:30.193188 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:30.193852 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:30.193853 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:30.194636 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:30.194638 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:30.194987 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:30.194988 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:30.195094 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:30.195509 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:51:30.195517 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:51:30.196904 (+ 1387us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:51:30.198399 (+ 1495us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:51:30.198405 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:51:30.198406 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:51:30.198601 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:51:30.198779 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:51:30.198780 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:51:30.198780 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:51:30.198873 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:51:30.199106 (+ 233us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:51:30.199112 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:51:30.199232 (+ 120us) client_negotiation.cc:769] Sending connection context
0807 08:51:30.199318 (+ 86us) client_negotiation.cc:240] Negotiation successful
0807 08:51:30.199460 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":74,"threads_started":1}
I20260807 08:51:30.200170 1821 heartbeater.cc:347] Connected to a master server at 127.4.159.190:46243
I20260807 08:51:30.200268 1821 heartbeater.cc:464] Registering TS with master...
I20260807 08:51:30.200426 1821 heartbeater.cc:511] Master 127.4.159.190:46243 requested a full tablet report, sending...
I20260807 08:51:30.200790 1323 ts_manager.cc:194] Registered new tserver with Master: bb79bb3fdbb94fe2b86bd969b9642d62 (127.4.159.130:38859)
I20260807 08:51:30.201301 1323 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.159.130@KRBTEST.COM'} at 127.4.159.130:50625
WARNING: no policy specified for kudu/127.4.159.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.159.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.159.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.131.keytab.
Entry for principal kudu/127.4.159.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.131.keytab.
I20260807 08:51:30.220038 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786092453486433-4734-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786092453486433-4734-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786092453486433-4734-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786092453486433-4734-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.159.131:0
--local_ip_for_outbound_sockets=127.4.159.131
--webserver_interface=127.4.159.131
--webserver_port=0
--tserver_master_addrs=127.4.159.190:46243
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.131.keytab
--principal=kudu/127.4.159.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:44215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:51:30.296761 1827 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:51:30.296926 1827 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:51:30.296947 1827 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:51:30.297986 1827 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:51:30.298050 1827 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:51:30.298082 1827 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.159.131
I20260807 08:51:30.299471 1827 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:44215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786092453486433-4734-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786092453486433-4734-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.131.keytab
--principal=kudu/127.4.159.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.159.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786092453486433-4734-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.159.131
--webserver_port=0
--tserver_master_addrs=127.4.159.190:46243
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1827
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.159.131
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786092453486433-4734-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:51:30.299628 1827 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:51:30.299793 1827 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:51:30.304106 1838 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:51:30.304137 1833 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:51:30.304160 1835 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:51:30.304373 1827 server_base.cc:1034] running on GCE node
I20260807 08:51:30.304538 1827 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:51:30.304739 1827 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:51:30.305867 1827 hybrid_clock.cc:648] HybridClock initialized: now 1786092690305854 us; error 29 us; skew 500 ppm
Aug 07 08:51:30 dist-test-slave-6zbq krb5kdc[1258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092690, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:51:30.308180 1827 init.cc:377] Logged in from keytab as kudu/127.4.159.131@KRBTEST.COM (short username kudu)
I20260807 08:51:30.308583 1827 webserver.cc:460] Webserver started at http://127.4.159.131:34481/ using document root <none> and password file <none>
I20260807 08:51:30.308765 1827 fs_manager.cc:359] Metadata directory not provided
I20260807 08:51:30.308822 1827 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:51:30.308939 1827 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:51:30.309872 1827 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786092453486433-4734-0/minicluster-data/ts-2/data/instance:
uuid: "7a353974be0c4daf99a7535b9903ff63"
format_stamp: "Formatted at 2026-08-07 08:51:30 on dist-test-slave-6zbq"
server_key: "33ea5560fbb559ac4dcfa0bd50a0450a"
server_key_iv: "4ac61b45b75c140c80fb83215b368482"
server_key_version: "encryptionkey@0"
I20260807 08:51:30.310148 1827 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786092453486433-4734-0/minicluster-data/ts-2/wal/instance:
uuid: "7a353974be0c4daf99a7535b9903ff63"
format_stamp: "Formatted at 2026-08-07 08:51:30 on dist-test-slave-6zbq"
server_key: "33ea5560fbb559ac4dcfa0bd50a0450a"
server_key_iv: "4ac61b45b75c140c80fb83215b368482"
server_key_version: "encryptionkey@0"
I20260807 08:51:30.311298 1827 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 08:51:30.311882 1849 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:51:30.312005 1827 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 08:51:30.312076 1827 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786092453486433-4734-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786092453486433-4734-0/minicluster-data/ts-2/wal
uuid: "7a353974be0c4daf99a7535b9903ff63"
format_stamp: "Formatted at 2026-08-07 08:51:30 on dist-test-slave-6zbq"
server_key: "33ea5560fbb559ac4dcfa0bd50a0450a"
server_key_iv: "4ac61b45b75c140c80fb83215b368482"
server_key_version: "encryptionkey@0"
I20260807 08:51:30.312137 1827 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786092453486433-4734-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786092453486433-4734-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786092453486433-4734-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:51:30.326977 1827 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:51:30.328817 1827 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:51:30.328935 1827 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:51:30.329105 1827 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:51:30.329355 1827 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:51:30.329387 1827 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:51:30.329406 1827 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:51:30.329427 1827 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:51:30.334573 1827 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.131:40521
I20260807 08:51:30.334612 2017 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.131:40521 every 8 connection(s)
I20260807 08:51:30.334869 1827 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786092453486433-4734-0/minicluster-data/ts-2/data/info.pb
Aug 07 08:51:30 dist-test-slave-6zbq krb5kdc[1258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092690, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.131@KRBTEST.COM for kudu/127.4.159.190@KRBTEST.COM
I20260807 08:51:30.342270 1386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:30.335478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.131:59463 (local address 127.4.159.190:46243)
0807 08:51:30.335532 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:30.335534 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:30.335694 (+ 160us) server_negotiation.cc:407] Connection header received
0807 08:51:30.336089 (+ 395us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:30.336089 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:30.336098 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:30.336146 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:51:30.336930 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:30.337416 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:30.338275 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:30.338495 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:30.338542 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:30.340308 (+ 1766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:51:30.340309 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:51:30.340309 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:51:30.340315 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:51:30.341333 (+ 1018us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:51:30.341638 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:51:30.341639 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:51:30.341639 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:51:30.341679 (+ 40us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:51:30.341842 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:51:30.341842 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:51:30.341842 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:51:30.341928 (+ 86us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:51:30.341969 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:30.342147 (+ 178us) server_negotiation.cc:299] Negotiation successful
0807 08:51:30.342199 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260807 08:51:30.342434 2020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:30.335446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:46243 (local address 127.4.159.131:59463)
0807 08:51:30.335585 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:30.335622 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 08:51:30.336018 (+ 396us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:51:30.336200 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:51:30.336203 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:51:30.336699 (+ 496us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:51:30.336867 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:30.336871 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:30.337484 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:30.337484 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:30.338217 (+ 733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:30.338219 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:30.338550 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:30.338551 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:30.338640 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:30.339030 (+ 390us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:51:30.339037 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:51:30.340229 (+ 1192us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:51:30.341391 (+ 1162us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:51:30.341395 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:51:30.341397 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:51:30.341585 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:51:30.341721 (+ 136us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:51:30.341721 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:51:30.341722 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:51:30.341802 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:51:30.341967 (+ 165us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:51:30.341969 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:51:30.342062 (+ 93us) client_negotiation.cc:769] Sending connection context
0807 08:51:30.342130 (+ 68us) client_negotiation.cc:240] Negotiation successful
0807 08:51:30.342241 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":66,"threads_started":1}
I20260807 08:51:30.342927 2018 heartbeater.cc:347] Connected to a master server at 127.4.159.190:46243
I20260807 08:51:30.343012 2018 heartbeater.cc:464] Registering TS with master...
I20260807 08:51:30.343165 2018 heartbeater.cc:511] Master 127.4.159.190:46243 requested a full tablet report, sending...
I20260807 08:51:30.343498 1323 ts_manager.cc:194] Registered new tserver with Master: 7a353974be0c4daf99a7535b9903ff63 (127.4.159.131:40521)
I20260807 08:51:30.343847 1323 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.159.131@KRBTEST.COM'} at 127.4.159.131:59463
I20260807 08:51:30.344053 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 1827
I20260807 08:51:30.344154 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786092453486433-4734-0/minicluster-data/ts-2/wal/instance
I20260807 08:51:30.344310 4734 external_mini_cluster.cc:1428] Setting key 19c07f4ad19f738667e58a977a8a6f20
I20260807 08:51:30.345078 4734 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 08:51:30.351475 1386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:30.346322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43088 (local address 127.4.159.190:46243)
0807 08:51:30.346390 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:30.346392 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:30.346488 (+ 96us) server_negotiation.cc:407] Connection header received
0807 08:51:30.346547 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:30.346547 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:30.346557 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:30.346596 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:51:30.347117 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:30.347572 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:30.348270 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:30.348464 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:30.348523 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:30.349464 (+ 941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:51:30.349465 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:51:30.349465 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:51:30.349470 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:51:30.350617 (+ 1147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:51:30.350894 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:51:30.350895 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:51:30.350895 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:51:30.350937 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:51:30.351090 (+ 153us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:51:30.351090 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:51:30.351090 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:51:30.351221 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:51:30.351291 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:30.351363 (+ 72us) server_negotiation.cc:299] Negotiation successful
0807 08:51:30.351420 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 08:51:30.352749 1323 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43088:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 08:51:30.354195 1323 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:51:30.359732 2028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:30.355998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.129:34869 (local address 127.0.0.1:44236)
0807 08:51:30.356153 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:30.356182 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0807 08:51:30.356229 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:51:30.356535 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:51:30.356538 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:51:30.356540 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:51:30.356587 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:30.356588 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:30.357589 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:30.357590 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:30.358464 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:30.358466 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:30.359463 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:30.359464 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:30.359543 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:30.359555 (+ 12us) client_negotiation.cc:769] Sending connection context
0807 08:51:30.359621 (+ 66us) client_negotiation.cc:240] Negotiation successful
0807 08:51:30.359668 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":44,"threads_started":1}
I20260807 08:51:30.359771 2032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:30.356122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44236 (local address 127.4.159.129:34869)
0807 08:51:30.356322 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:30.356326 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:30.356335 (+ 9us) server_negotiation.cc:407] Connection header received
0807 08:51:30.356371 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:30.356372 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:30.356385 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:30.356451 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:51:30.356657 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:30.357497 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:30.358557 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:30.359383 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:30.359457 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:30.359514 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:30.359639 (+ 125us) server_negotiation.cc:299] Negotiation successful
0807 08:51:30.359693 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":106,"threads_started":1}
I20260807 08:51:30.360247 2030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:30.356033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.130:38859 (local address 127.0.0.1:59162)
0807 08:51:30.356550 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:30.356558 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0807 08:51:30.356612 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:51:30.356880 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:51:30.356881 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:51:30.356883 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:51:30.356933 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:30.356935 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:30.358266 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:30.358267 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:30.359201 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:30.359203 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:30.360017 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:30.360018 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:30.360085 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:30.360088 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 08:51:30.360137 (+ 49us) client_negotiation.cc:240] Negotiation successful
0807 08:51:30.360173 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"mutex_wait_us":36,"thread_start_us":60,"threads_started":1}
I20260807 08:51:30.360519 2035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:30.356350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59162 (local address 127.4.159.130:38859)
0807 08:51:30.356513 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:30.356516 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:30.356622 (+ 106us) server_negotiation.cc:407] Connection header received
0807 08:51:30.356683 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:30.356685 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:30.356700 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:30.356763 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:51:30.357025 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:30.358159 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:30.359274 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:30.359947 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:30.360009 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:30.360059 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:30.360400 (+ 341us) server_negotiation.cc:299] Negotiation successful
0807 08:51:30.360443 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":66,"threads_started":1}
I20260807 08:51:30.360529 2034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:30.356322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38204 (local address 127.4.159.131:40521)
0807 08:51:30.356477 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:30.356480 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:30.356751 (+ 271us) server_negotiation.cc:407] Connection header received
0807 08:51:30.356808 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:30.356810 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:30.356824 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:30.356890 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:51:30.357111 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:30.358149 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:30.359474 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:30.360128 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:30.360200 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:30.360249 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:30.360431 (+ 182us) server_negotiation.cc:299] Negotiation successful
0807 08:51:30.360463 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":84,"threads_started":1}
I20260807 08:51:30.360522 2033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:30.356169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.131:40521 (local address 127.0.0.1:38204)
0807 08:51:30.356686 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:30.356695 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 08:51:30.356751 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:51:30.356948 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:51:30.356949 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:51:30.356951 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:51:30.357041 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:30.357043 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:30.358268 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:30.358269 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:30.359390 (+ 1121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:30.359392 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:30.360283 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:30.360284 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:30.360349 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:30.360353 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 08:51:30.360407 (+ 54us) client_negotiation.cc:240] Negotiation successful
0807 08:51:30.360443 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":502,"mutex_wait_us":4,"thread_start_us":59,"threads_started":1}
I20260807 08:51:30.360718 1519 tablet_service.cc:1467] Processing CreateTablet for tablet c2f407fe98da470f80c8f4e055a965e1 (DEFAULT_TABLE table=test-table [id=1aefd2cf09ac4bf1bd817df27106e12b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:51:30.361091 1519 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2f407fe98da470f80c8f4e055a965e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:51:30.361093 1721 tablet_service.cc:1467] Processing CreateTablet for tablet c2f407fe98da470f80c8f4e055a965e1 (DEFAULT_TABLE table=test-table [id=1aefd2cf09ac4bf1bd817df27106e12b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:51:30.361145 1913 tablet_service.cc:1467] Processing CreateTablet for tablet c2f407fe98da470f80c8f4e055a965e1 (DEFAULT_TABLE table=test-table [id=1aefd2cf09ac4bf1bd817df27106e12b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:51:30.361405 1721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2f407fe98da470f80c8f4e055a965e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:51:30.361438 1913 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2f407fe98da470f80c8f4e055a965e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:51:30.364096 2037 tablet_bootstrap.cc:492] T c2f407fe98da470f80c8f4e055a965e1 P 7a353974be0c4daf99a7535b9903ff63: Bootstrap starting.
I20260807 08:51:30.364517 2041 tablet_bootstrap.cc:492] T c2f407fe98da470f80c8f4e055a965e1 P bb79bb3fdbb94fe2b86bd969b9642d62: Bootstrap starting.
I20260807 08:51:30.364727 2037 tablet_bootstrap.cc:654] T c2f407fe98da470f80c8f4e055a965e1 P 7a353974be0c4daf99a7535b9903ff63: Neither blocks nor log segments found. Creating new log.
I20260807 08:51:30.365031 2041 tablet_bootstrap.cc:654] T c2f407fe98da470f80c8f4e055a965e1 P bb79bb3fdbb94fe2b86bd969b9642d62: Neither blocks nor log segments found. Creating new log.
I20260807 08:51:30.365195 2037 log.cc:824] T c2f407fe98da470f80c8f4e055a965e1 P 7a353974be0c4daf99a7535b9903ff63: Log is configured to *not* fsync() on all Append() calls
I20260807 08:51:30.365509 2041 log.cc:824] T c2f407fe98da470f80c8f4e055a965e1 P bb79bb3fdbb94fe2b86bd969b9642d62: Log is configured to *not* fsync() on all Append() calls
I20260807 08:51:30.365855 2037 tablet_bootstrap.cc:492] T c2f407fe98da470f80c8f4e055a965e1 P 7a353974be0c4daf99a7535b9903ff63: No bootstrap required, opened a new log
I20260807 08:51:30.365831 2039 tablet_bootstrap.cc:492] T c2f407fe98da470f80c8f4e055a965e1 P 25d93c2f00e544eba4c891390e740d56: Bootstrap starting.
I20260807 08:51:30.365913 2037 ts_tablet_manager.cc:1397] T c2f407fe98da470f80c8f4e055a965e1 P 7a353974be0c4daf99a7535b9903ff63: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 08:51:30.366164 2041 tablet_bootstrap.cc:492] T c2f407fe98da470f80c8f4e055a965e1 P bb79bb3fdbb94fe2b86bd969b9642d62: No bootstrap required, opened a new log
I20260807 08:51:30.366218 2041 ts_tablet_manager.cc:1397] T c2f407fe98da470f80c8f4e055a965e1 P bb79bb3fdbb94fe2b86bd969b9642d62: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 08:51:30.367241 2037 raft_consensus.cc:348] T c2f407fe98da470f80c8f4e055a965e1 P 7a353974be0c4daf99a7535b9903ff63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25d93c2f00e544eba4c891390e740d56" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 34869 } } peers { permanent_uuid: "7a353974be0c4daf99a7535b9903ff63" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 40521 } } peers { permanent_uuid: "bb79bb3fdbb94fe2b86bd969b9642d62" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 38859 } }
I20260807 08:51:30.367400 2037 raft_consensus.cc:374] T c2f407fe98da470f80c8f4e055a965e1 P 7a353974be0c4daf99a7535b9903ff63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:51:30.367509 2037 raft_consensus.cc:729] T c2f407fe98da470f80c8f4e055a965e1 P 7a353974be0c4daf99a7535b9903ff63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a353974be0c4daf99a7535b9903ff63, State: Initialized, Role: FOLLOWER
I20260807 08:51:30.367640 2037 consensus_queue.cc:260] T c2f407fe98da470f80c8f4e055a965e1 P 7a353974be0c4daf99a7535b9903ff63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25d93c2f00e544eba4c891390e740d56" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 34869 } } peers { permanent_uuid: "7a353974be0c4daf99a7535b9903ff63" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 40521 } } peers { permanent_uuid: "bb79bb3fdbb94fe2b86bd969b9642d62" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 38859 } }
I20260807 08:51:30.367549 2041 raft_consensus.cc:348] T c2f407fe98da470f80c8f4e055a965e1 P bb79bb3fdbb94fe2b86bd969b9642d62 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25d93c2f00e544eba4c891390e740d56" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 34869 } } peers { permanent_uuid: "7a353974be0c4daf99a7535b9903ff63" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 40521 } } peers { permanent_uuid: "bb79bb3fdbb94fe2b86bd969b9642d62" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 38859 } }
I20260807 08:51:30.367712 2041 raft_consensus.cc:374] T c2f407fe98da470f80c8f4e055a965e1 P bb79bb3fdbb94fe2b86bd969b9642d62 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:51:30.367748 2041 raft_consensus.cc:729] T c2f407fe98da470f80c8f4e055a965e1 P bb79bb3fdbb94fe2b86bd969b9642d62 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb79bb3fdbb94fe2b86bd969b9642d62, State: Initialized, Role: FOLLOWER
I20260807 08:51:30.367882 2037 ts_tablet_manager.cc:1428] T c2f407fe98da470f80c8f4e055a965e1 P 7a353974be0c4daf99a7535b9903ff63: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 08:51:30.367901 2018 heartbeater.cc:503] Master 127.4.159.190:46243 was elected leader, sending a full tablet report...
I20260807 08:51:30.367877 2041 consensus_queue.cc:260] T c2f407fe98da470f80c8f4e055a965e1 P bb79bb3fdbb94fe2b86bd969b9642d62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25d93c2f00e544eba4c891390e740d56" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 34869 } } peers { permanent_uuid: "7a353974be0c4daf99a7535b9903ff63" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 40521 } } peers { permanent_uuid: "bb79bb3fdbb94fe2b86bd969b9642d62" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 38859 } }
I20260807 08:51:30.368153 1821 heartbeater.cc:503] Master 127.4.159.190:46243 was elected leader, sending a full tablet report...
I20260807 08:51:30.368383 2041 ts_tablet_manager.cc:1428] T c2f407fe98da470f80c8f4e055a965e1 P bb79bb3fdbb94fe2b86bd969b9642d62: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 08:51:30.369024 2039 tablet_bootstrap.cc:654] T c2f407fe98da470f80c8f4e055a965e1 P 25d93c2f00e544eba4c891390e740d56: Neither blocks nor log segments found. Creating new log.
I20260807 08:51:30.369477 2039 log.cc:824] T c2f407fe98da470f80c8f4e055a965e1 P 25d93c2f00e544eba4c891390e740d56: Log is configured to *not* fsync() on all Append() calls
I20260807 08:51:30.370054 2039 tablet_bootstrap.cc:492] T c2f407fe98da470f80c8f4e055a965e1 P 25d93c2f00e544eba4c891390e740d56: No bootstrap required, opened a new log
I20260807 08:51:30.370103 2039 ts_tablet_manager.cc:1397] T c2f407fe98da470f80c8f4e055a965e1 P 25d93c2f00e544eba4c891390e740d56: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260807 08:51:30.371450 2039 raft_consensus.cc:348] T c2f407fe98da470f80c8f4e055a965e1 P 25d93c2f00e544eba4c891390e740d56 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25d93c2f00e544eba4c891390e740d56" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 34869 } } peers { permanent_uuid: "7a353974be0c4daf99a7535b9903ff63" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 40521 } } peers { permanent_uuid: "bb79bb3fdbb94fe2b86bd969b9642d62" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 38859 } }
I20260807 08:51:30.371573 2039 raft_consensus.cc:374] T c2f407fe98da470f80c8f4e055a965e1 P 25d93c2f00e544eba4c891390e740d56 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:51:30.371605 2039 raft_consensus.cc:729] T c2f407fe98da470f80c8f4e055a965e1 P 25d93c2f00e544eba4c891390e740d56 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25d93c2f00e544eba4c891390e740d56, State: Initialized, Role: FOLLOWER
I20260807 08:51:30.371708 2039 consensus_queue.cc:260] T c2f407fe98da470f80c8f4e055a965e1 P 25d93c2f00e544eba4c891390e740d56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25d93c2f00e544eba4c891390e740d56" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 34869 } } peers { permanent_uuid: "7a353974be0c4daf99a7535b9903ff63" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 40521 } } peers { permanent_uuid: "bb79bb3fdbb94fe2b86bd969b9642d62" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 38859 } }
I20260807 08:51:30.371932 2039 ts_tablet_manager.cc:1428] T c2f407fe98da470f80c8f4e055a965e1 P 25d93c2f00e544eba4c891390e740d56: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 08:51:30.371976 1622 heartbeater.cc:503] Master 127.4.159.190:46243 was elected leader, sending a full tablet report...
W20260807 08:51:30.441496 1822 tablet.cc:2369] T c2f407fe98da470f80c8f4e055a965e1 P bb79bb3fdbb94fe2b86bd969b9642d62: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 08:51:30.548296 1623 tablet.cc:2369] T c2f407fe98da470f80c8f4e055a965e1 P 25d93c2f00e544eba4c891390e740d56: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 08:51:30.585381 2019 tablet.cc:2369] T c2f407fe98da470f80c8f4e055a965e1 P 7a353974be0c4daf99a7535b9903ff63: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 08:51:30.595438 2049 raft_consensus.cc:482] T c2f407fe98da470f80c8f4e055a965e1 P 25d93c2f00e544eba4c891390e740d56 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:51:30.595544 2049 raft_consensus.cc:504] T c2f407fe98da470f80c8f4e055a965e1 P 25d93c2f00e544eba4c891390e740d56 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25d93c2f00e544eba4c891390e740d56" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 34869 } } peers { permanent_uuid: "7a353974be0c4daf99a7535b9903ff63" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 40521 } } peers { permanent_uuid: "bb79bb3fdbb94fe2b86bd969b9642d62" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 38859 } }
I20260807 08:51:30.595858 2049 leader_election.cc:288] T c2f407fe98da470f80c8f4e055a965e1 P 25d93c2f00e544eba4c891390e740d56 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7a353974be0c4daf99a7535b9903ff63 (127.4.159.131:40521), bb79bb3fdbb94fe2b86bd969b9642d62 (127.4.159.130:38859)
I20260807 08:51:30.598619 2035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:30.596132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.129:55017 (local address 127.4.159.130:38859)
0807 08:51:30.596184 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:30.596186 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:30.596261 (+ 75us) server_negotiation.cc:407] Connection header received
0807 08:51:30.596314 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:30.596315 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:30.596325 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:30.596366 (+ 41us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:51:30.596516 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:30.596974 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:30.597913 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:30.598289 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:30.598373 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:30.598383 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:30.598514 (+ 131us) server_negotiation.cc:299] Negotiation successful
0807 08:51:30.598549 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260807 08:51:30.598637 2052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:30.596040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.130:38859 (local address 127.4.159.129:55017)
0807 08:51:30.596191 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:30.596201 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 08:51:30.596256 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:51:30.596375 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:51:30.596376 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:51:30.596378 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:51:30.596456 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:30.596458 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:30.597042 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:30.597043 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:30.597828 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:30.597831 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:30.598384 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:30.598385 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:30.598453 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:30.598456 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 08:51:30.598501 (+ 45us) client_negotiation.cc:240] Negotiation successful
0807 08:51:30.598539 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"mutex_wait_us":3,"thread_start_us":79,"threads_started":1}
I20260807 08:51:30.598664 2051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:30.596014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.131:40521 (local address 127.4.159.129:36775)
0807 08:51:30.596139 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:30.596149 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 08:51:30.596205 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:51:30.596336 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:51:30.596336 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:51:30.596338 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:51:30.596411 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:30.596413 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:30.597012 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:30.597013 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:30.597826 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:30.597829 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:30.598416 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:30.598417 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:30.598481 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:30.598484 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 08:51:30.598528 (+ 44us) client_negotiation.cc:240] Negotiation successful
0807 08:51:30.598568 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"mutex_wait_us":37,"thread_start_us":60,"threads_started":1}
I20260807 08:51:30.598642 2034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:30.596097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.129:36775 (local address 127.4.159.131:40521)
0807 08:51:30.596143 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:30.596200 (+ 57us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:30.596209 (+ 9us) server_negotiation.cc:407] Connection header received
0807 08:51:30.596264 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:30.596265 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:30.596274 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:30.596325 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:51:30.596470 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:30.596941 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:30.597916 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:30.598324 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:30.598408 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:30.598417 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:30.598543 (+ 126us) server_negotiation.cc:299] Negotiation successful
0807 08:51:30.598575 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 08:51:30.598910 1752 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2f407fe98da470f80c8f4e055a965e1" candidate_uuid: "25d93c2f00e544eba4c891390e740d56" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb79bb3fdbb94fe2b86bd969b9642d62" is_pre_election: true
I20260807 08:51:30.599051 1752 raft_consensus.cc:2454] T c2f407fe98da470f80c8f4e055a965e1 P bb79bb3fdbb94fe2b86bd969b9642d62 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 25d93c2f00e544eba4c891390e740d56 in term 0.
I20260807 08:51:30.599068 1945 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2f407fe98da470f80c8f4e055a965e1" candidate_uuid: "25d93c2f00e544eba4c891390e740d56" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a353974be0c4daf99a7535b9903ff63" is_pre_election: true
I20260807 08:51:30.599208 1945 raft_consensus.cc:2454] T c2f407fe98da470f80c8f4e055a965e1 P 7a353974be0c4daf99a7535b9903ff63 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 25d93c2f00e544eba4c891390e740d56 in term 0.
I20260807 08:51:30.599260 1457 leader_election.cc:302] T c2f407fe98da470f80c8f4e055a965e1 P 25d93c2f00e544eba4c891390e740d56 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25d93c2f00e544eba4c891390e740d56, bb79bb3fdbb94fe2b86bd969b9642d62; no voters:
I20260807 08:51:30.599386 2049 raft_consensus.cc:2781] T c2f407fe98da470f80c8f4e055a965e1 P 25d93c2f00e544eba4c891390e740d56 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:51:30.599436 2049 raft_consensus.cc:482] T c2f407fe98da470f80c8f4e055a965e1 P 25d93c2f00e544eba4c891390e740d56 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:51:30.599457 2049 raft_consensus.cc:3037] T c2f407fe98da470f80c8f4e055a965e1 P 25d93c2f00e544eba4c891390e740d56 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:51:30.600148 2049 raft_consensus.cc:504] T c2f407fe98da470f80c8f4e055a965e1 P 25d93c2f00e544eba4c891390e740d56 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25d93c2f00e544eba4c891390e740d56" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 34869 } } peers { permanent_uuid: "7a353974be0c4daf99a7535b9903ff63" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 40521 } } peers { permanent_uuid: "bb79bb3fdbb94fe2b86bd969b9642d62" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 38859 } }
I20260807 08:51:30.600261 2049 leader_election.cc:288] T c2f407fe98da470f80c8f4e055a965e1 P 25d93c2f00e544eba4c891390e740d56 [CANDIDATE]: Term 1 election: Requested vote from peers 7a353974be0c4daf99a7535b9903ff63 (127.4.159.131:40521), bb79bb3fdbb94fe2b86bd969b9642d62 (127.4.159.130:38859)
I20260807 08:51:30.600431 1945 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2f407fe98da470f80c8f4e055a965e1" candidate_uuid: "25d93c2f00e544eba4c891390e740d56" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a353974be0c4daf99a7535b9903ff63"
I20260807 08:51:30.600457 1752 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2f407fe98da470f80c8f4e055a965e1" candidate_uuid: "25d93c2f00e544eba4c891390e740d56" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb79bb3fdbb94fe2b86bd969b9642d62"
I20260807 08:51:30.600517 1945 raft_consensus.cc:3037] T c2f407fe98da470f80c8f4e055a965e1 P 7a353974be0c4daf99a7535b9903ff63 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:51:30.600517 1752 raft_consensus.cc:3037] T c2f407fe98da470f80c8f4e055a965e1 P bb79bb3fdbb94fe2b86bd969b9642d62 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:51:30.601372 1752 raft_consensus.cc:2454] T c2f407fe98da470f80c8f4e055a965e1 P bb79bb3fdbb94fe2b86bd969b9642d62 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25d93c2f00e544eba4c891390e740d56 in term 1.
I20260807 08:51:30.601372 1945 raft_consensus.cc:2454] T c2f407fe98da470f80c8f4e055a965e1 P 7a353974be0c4daf99a7535b9903ff63 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25d93c2f00e544eba4c891390e740d56 in term 1.
I20260807 08:51:30.601562 1454 leader_election.cc:302] T c2f407fe98da470f80c8f4e055a965e1 P 25d93c2f00e544eba4c891390e740d56 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25d93c2f00e544eba4c891390e740d56, 7a353974be0c4daf99a7535b9903ff63; no voters:
I20260807 08:51:30.601712 2049 raft_consensus.cc:2781] T c2f407fe98da470f80c8f4e055a965e1 P 25d93c2f00e544eba4c891390e740d56 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:51:30.601835 2049 raft_consensus.cc:686] T c2f407fe98da470f80c8f4e055a965e1 P 25d93c2f00e544eba4c891390e740d56 [term 1 LEADER]: Becoming Leader. State: Replica: 25d93c2f00e544eba4c891390e740d56, State: Running, Role: LEADER
I20260807 08:51:30.601910 2049 consensus_queue.cc:237] T c2f407fe98da470f80c8f4e055a965e1 P 25d93c2f00e544eba4c891390e740d56 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25d93c2f00e544eba4c891390e740d56" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 34869 } } peers { permanent_uuid: "7a353974be0c4daf99a7535b9903ff63" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 40521 } } peers { permanent_uuid: "bb79bb3fdbb94fe2b86bd969b9642d62" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 38859 } }
I20260807 08:51:30.602660 1322 catalog_manager.cc:5495] T c2f407fe98da470f80c8f4e055a965e1 P 25d93c2f00e544eba4c891390e740d56 reported cstate change: term changed from 0 to 1, leader changed from <none> to 25d93c2f00e544eba4c891390e740d56 (127.4.159.129). New cstate: current_term: 1 leader_uuid: "25d93c2f00e544eba4c891390e740d56" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25d93c2f00e544eba4c891390e740d56" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 34869 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a353974be0c4daf99a7535b9903ff63" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 40521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb79bb3fdbb94fe2b86bd969b9642d62" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 38859 } health_report { overall_health: UNKNOWN } } }
I20260807 08:51:30.638973 2032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:30.637007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44250 (local address 127.4.159.129:34869)
0807 08:51:30.637049 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:30.637050 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:30.637058 (+ 8us) server_negotiation.cc:407] Connection header received
0807 08:51:30.637103 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:30.637104 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:30.637123 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:30.637184 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:51:30.637280 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:30.637752 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:30.638440 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:30.638613 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:30.638661 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:30.638759 (+ 98us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:51:30.638845 (+ 86us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:51:30.638895 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:30.638907 (+ 12us) server_negotiation.cc:299] Negotiation successful
0807 08:51:30.638922 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 08:51:30.640525 1752 raft_consensus.cc:1261] T c2f407fe98da470f80c8f4e055a965e1 P bb79bb3fdbb94fe2b86bd969b9642d62 [term 1 FOLLOWER]: Refusing update from remote peer 25d93c2f00e544eba4c891390e740d56: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:51:30.640530 1945 raft_consensus.cc:1261] T c2f407fe98da470f80c8f4e055a965e1 P 7a353974be0c4daf99a7535b9903ff63 [term 1 FOLLOWER]: Refusing update from remote peer 25d93c2f00e544eba4c891390e740d56: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:51:30.640820 2049 consensus_queue.cc:1035] T c2f407fe98da470f80c8f4e055a965e1 P 25d93c2f00e544eba4c891390e740d56 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb79bb3fdbb94fe2b86bd969b9642d62" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 38859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:51:30.640954 2053 consensus_queue.cc:1035] T c2f407fe98da470f80c8f4e055a965e1 P 25d93c2f00e544eba4c891390e740d56 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a353974be0c4daf99a7535b9903ff63" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 40521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:51:30.642993 2057 mvcc.cc:205] Tried to move back new op lower bound from 7315835660861448192 to 7315835660706013184. Current Snapshot: MvccSnapshot[applied={T|T < 7315835660861448192}]
I20260807 08:51:30.643695 2060 mvcc.cc:205] Tried to move back new op lower bound from 7315835660861448192 to 7315835660706013184. Current Snapshot: MvccSnapshot[applied={T|T < 7315835660861448192}]
I20260807 08:51:30.644086 2059 mvcc.cc:205] Tried to move back new op lower bound from 7315835660861448192 to 7315835660706013184. Current Snapshot: MvccSnapshot[applied={T|T < 7315835660861448192}]
I20260807 08:51:30.645781 1322 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43088:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:51:30.645872 1322 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43088:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:51:30.646605 1322 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b59b19d11e64484f8c4a45f30423dc68: Sending DeleteTablet for 3 replicas of tablet c2f407fe98da470f80c8f4e055a965e1
I20260807 08:51:30.646880 1519 tablet_service.cc:1514] Processing DeleteTablet for tablet c2f407fe98da470f80c8f4e055a965e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:51:30 UTC) from {username='kudu', principal='kudu/127.4.159.190@KRBTEST.COM'} at 127.0.0.1:44236
I20260807 08:51:30.646932 1913 tablet_service.cc:1514] Processing DeleteTablet for tablet c2f407fe98da470f80c8f4e055a965e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:51:30 UTC) from {username='kudu', principal='kudu/127.4.159.190@KRBTEST.COM'} at 127.0.0.1:38204
I20260807 08:51:30.647097 2073 tablet_replica.cc:330] T c2f407fe98da470f80c8f4e055a965e1 P 25d93c2f00e544eba4c891390e740d56: stopping tablet replica
I20260807 08:51:30.647195 2074 tablet_replica.cc:330] T c2f407fe98da470f80c8f4e055a965e1 P 7a353974be0c4daf99a7535b9903ff63: stopping tablet replica
I20260807 08:51:30.647228 2073 raft_consensus.cc:2229] T c2f407fe98da470f80c8f4e055a965e1 P 25d93c2f00e544eba4c891390e740d56 [term 1 LEADER]: Raft consensus shutting down.
I20260807 08:51:30.647300 2074 raft_consensus.cc:2229] T c2f407fe98da470f80c8f4e055a965e1 P 7a353974be0c4daf99a7535b9903ff63 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:51:30.647310 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 1425
I20260807 08:51:30.647408 2073 raft_consensus.cc:2258] T c2f407fe98da470f80c8f4e055a965e1 P 25d93c2f00e544eba4c891390e740d56 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:51:30.647423 2074 raft_consensus.cc:2258] T c2f407fe98da470f80c8f4e055a965e1 P 7a353974be0c4daf99a7535b9903ff63 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:51:30.650323 2074 ts_tablet_manager.cc:1905] T c2f407fe98da470f80c8f4e055a965e1 P 7a353974be0c4daf99a7535b9903ff63: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:51:30.651682 2074 ts_tablet_manager.cc:1918] T c2f407fe98da470f80c8f4e055a965e1 P 7a353974be0c4daf99a7535b9903ff63: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:51:30.651731 2074 log.cc:1196] T c2f407fe98da470f80c8f4e055a965e1 P 7a353974be0c4daf99a7535b9903ff63: Deleting WAL directory at /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786092453486433-4734-0/minicluster-data/ts-2/wal/wals/c2f407fe98da470f80c8f4e055a965e1
I20260807 08:51:30.651964 2074 ts_tablet_manager.cc:1939] T c2f407fe98da470f80c8f4e055a965e1 P 7a353974be0c4daf99a7535b9903ff63: Deleting consensus metadata
I20260807 08:51:30.652318 1298 catalog_manager.cc:4841] TS 7a353974be0c4daf99a7535b9903ff63 (127.4.159.131:40521): tablet c2f407fe98da470f80c8f4e055a965e1 (table test-table [id=1aefd2cf09ac4bf1bd817df27106e12b]) successfully deleted
I20260807 08:51:30.654143 1721 tablet_service.cc:1514] Processing DeleteTablet for tablet c2f407fe98da470f80c8f4e055a965e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:51:30 UTC) from {username='kudu', principal='kudu/127.4.159.190@KRBTEST.COM'} at 127.0.0.1:59162
I20260807 08:51:30.654333 2075 tablet_replica.cc:330] T c2f407fe98da470f80c8f4e055a965e1 P bb79bb3fdbb94fe2b86bd969b9642d62: stopping tablet replica
I20260807 08:51:30.654455 2075 raft_consensus.cc:2229] T c2f407fe98da470f80c8f4e055a965e1 P bb79bb3fdbb94fe2b86bd969b9642d62 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:51:30.654552 2075 raft_consensus.cc:2258] T c2f407fe98da470f80c8f4e055a965e1 P bb79bb3fdbb94fe2b86bd969b9642d62 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:51:30.655593 2075 ts_tablet_manager.cc:1905] T c2f407fe98da470f80c8f4e055a965e1 P bb79bb3fdbb94fe2b86bd969b9642d62: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:51:30.656670 2075 ts_tablet_manager.cc:1918] T c2f407fe98da470f80c8f4e055a965e1 P bb79bb3fdbb94fe2b86bd969b9642d62: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:51:30.656734 2075 log.cc:1196] T c2f407fe98da470f80c8f4e055a965e1 P bb79bb3fdbb94fe2b86bd969b9642d62: Deleting WAL directory at /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786092453486433-4734-0/minicluster-data/ts-1/wal/wals/c2f407fe98da470f80c8f4e055a965e1
I20260807 08:51:30.656945 2075 ts_tablet_manager.cc:1939] T c2f407fe98da470f80c8f4e055a965e1 P bb79bb3fdbb94fe2b86bd969b9642d62: Deleting consensus metadata
I20260807 08:51:30.657290 1304 catalog_manager.cc:4841] TS bb79bb3fdbb94fe2b86bd969b9642d62 (127.4.159.130:38859): tablet c2f407fe98da470f80c8f4e055a965e1 (table test-table [id=1aefd2cf09ac4bf1bd817df27106e12b]) successfully deleted
W20260807 08:51:30.659276 1298 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.159.129:34869 (error 108)
I20260807 08:51:30.659505 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 1627
I20260807 08:51:30.663822 2030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:30.659557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.129:34869 (local address 127.0.0.1:44252)
0807 08:51:30.663729 (+ 4172us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:30.663784 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.129:34869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4145}
W20260807 08:51:30.664482 1298 catalog_manager.cc:4584] TS 25d93c2f00e544eba4c891390e740d56 (127.4.159.129:34869): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c2f407fe98da470f80c8f4e055a965e1: Network error: Client connection negotiation failed: client connection to 127.4.159.129:34869: connect: Connection refused (error 111)
I20260807 08:51:30.671074 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 1827
I20260807 08:51:30.681716 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 1278
2026-08-07T08:51:30Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2160 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-07T08:51:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:51:30Z Disabled control of system clock
I20260807 08:51:30.711274 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786092453486433-4734-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786092453486433-4734-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786092453486433-4734-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.159.190:40057
--webserver_interface=127.4.159.190
--webserver_port=0
--builtin_ntp_servers=127.4.159.148:33687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.159.190:40057
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 08:51:30.789789 2081 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:51:30.789987 2081 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:51:30.790006 2081 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:51:30.791131 2081 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:51:30.791172 2081 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:51:30.791183 2081 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:51:30.791194 2081 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:51:30.791204 2081 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:51:30.792480 2081 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:33687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786092453486433-4734-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786092453486433-4734-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.159.190:40057
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.159.190:40057
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.159.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2081
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786092453486433-4734-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:51:30.792659 2081 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:51:30.792894 2081 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:51:30.797315 2088 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:51:30.797297 2091 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:51:30.797446 2081 server_base.cc:1034] running on GCE node
W20260807 08:51:30.797286 2089 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:51:30.797744 2081 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:51:30.798007 2081 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:51:30.799136 2081 hybrid_clock.cc:648] HybridClock initialized: now 1786092690799122 us; error 29 us; skew 500 ppm
I20260807 08:51:30.800278 2081 webserver.cc:460] Webserver started at http://127.4.159.190:35215/ using document root <none> and password file <none>
I20260807 08:51:30.800508 2081 fs_manager.cc:359] Metadata directory not provided
I20260807 08:51:30.800560 2081 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:51:30.800683 2081 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:51:30.801697 2081 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786092453486433-4734-0/minicluster-data/master-0/data/instance:
uuid: "fbe3ba186c684319918b631b95983caa"
format_stamp: "Formatted at 2026-08-07 08:51:30 on dist-test-slave-6zbq"
server_key: "47abc0fc2d313dc7386631120e9a9e57"
server_key_iv: "0ef199939275620cfb45bfaf9c69e214"
server_key_version: "encryptionkey@0"
I20260807 08:51:30.802023 2081 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786092453486433-4734-0/minicluster-data/master-0/wal/instance:
uuid: "fbe3ba186c684319918b631b95983caa"
format_stamp: "Formatted at 2026-08-07 08:51:30 on dist-test-slave-6zbq"
server_key: "47abc0fc2d313dc7386631120e9a9e57"
server_key_iv: "0ef199939275620cfb45bfaf9c69e214"
server_key_version: "encryptionkey@0"
I20260807 08:51:30.803285 2081 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 08:51:30.803910 2099 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:51:30.804067 2081 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 08:51:30.804149 2081 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786092453486433-4734-0/minicluster-data/master-0/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786092453486433-4734-0/minicluster-data/master-0/wal
uuid: "fbe3ba186c684319918b631b95983caa"
format_stamp: "Formatted at 2026-08-07 08:51:30 on dist-test-slave-6zbq"
server_key: "47abc0fc2d313dc7386631120e9a9e57"
server_key_iv: "0ef199939275620cfb45bfaf9c69e214"
server_key_version: "encryptionkey@0"
I20260807 08:51:30.804208 2081 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786092453486433-4734-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786092453486433-4734-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786092453486433-4734-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:51:30.815915 2081 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:51:30.816160 2081 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:51:30.816287 2081 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:51:30.819967 2081 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.190:40057
I20260807 08:51:30.819998 2177 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.190:40057 every 8 connection(s)
I20260807 08:51:30.820288 2081 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
I20260807 08:51:30.821190 2180 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:51:30.823400 2180 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fbe3ba186c684319918b631b95983caa: Bootstrap starting.
I20260807 08:51:30.823930 2180 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fbe3ba186c684319918b631b95983caa: Neither blocks nor log segments found. Creating new log.
I20260807 08:51:30.824208 2180 log.cc:824] T 00000000000000000000000000000000 P fbe3ba186c684319918b631b95983caa: Log is configured to *not* fsync() on all Append() calls
I20260807 08:51:30.824798 2180 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fbe3ba186c684319918b631b95983caa: No bootstrap required, opened a new log
I20260807 08:51:30.825048 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 2081
I20260807 08:51:30.825147 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786092453486433-4734-0/minicluster-data/master-0/wal/instance
I20260807 08:51:30.825311 4734 external_mini_cluster.cc:1428] Setting key 6d81ead6071b17ed124c1b3824b0b47d
I20260807 08:51:30.826925 2180 raft_consensus.cc:348] T 00000000000000000000000000000000 P fbe3ba186c684319918b631b95983caa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbe3ba186c684319918b631b95983caa" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 40057 } }
I20260807 08:51:30.827059 2180 raft_consensus.cc:374] T 00000000000000000000000000000000 P fbe3ba186c684319918b631b95983caa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:51:30.827090 2180 raft_consensus.cc:729] T 00000000000000000000000000000000 P fbe3ba186c684319918b631b95983caa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbe3ba186c684319918b631b95983caa, State: Initialized, Role: FOLLOWER
I20260807 08:51:30.827239 2180 consensus_queue.cc:260] T 00000000000000000000000000000000 P fbe3ba186c684319918b631b95983caa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbe3ba186c684319918b631b95983caa" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 40057 } }
I20260807 08:51:30.827324 2180 raft_consensus.cc:388] T 00000000000000000000000000000000 P fbe3ba186c684319918b631b95983caa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:51:30.827356 2180 raft_consensus.cc:482] T 00000000000000000000000000000000 P fbe3ba186c684319918b631b95983caa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:51:30.827404 2180 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fbe3ba186c684319918b631b95983caa [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:51:30.827893 2180 raft_consensus.cc:504] T 00000000000000000000000000000000 P fbe3ba186c684319918b631b95983caa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbe3ba186c684319918b631b95983caa" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 40057 } }
I20260807 08:51:30.827986 2180 leader_election.cc:302] T 00000000000000000000000000000000 P fbe3ba186c684319918b631b95983caa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fbe3ba186c684319918b631b95983caa; no voters:
I20260807 08:51:30.828146 2180 leader_election.cc:288] T 00000000000000000000000000000000 P fbe3ba186c684319918b631b95983caa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:51:30.828209 2190 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fbe3ba186c684319918b631b95983caa [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:51:30.828325 2190 raft_consensus.cc:686] T 00000000000000000000000000000000 P fbe3ba186c684319918b631b95983caa [term 1 LEADER]: Becoming Leader. State: Replica: fbe3ba186c684319918b631b95983caa, State: Running, Role: LEADER
I20260807 08:51:30.828406 2180 sys_catalog.cc:564] T 00000000000000000000000000000000 P fbe3ba186c684319918b631b95983caa [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:51:30.828461 2190 consensus_queue.cc:237] T 00000000000000000000000000000000 P fbe3ba186c684319918b631b95983caa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbe3ba186c684319918b631b95983caa" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 40057 } }
I20260807 08:51:30.829865 2191 sys_catalog.cc:455] T 00000000000000000000000000000000 P fbe3ba186c684319918b631b95983caa [sys.catalog]: SysCatalogTable state changed. Reason: New leader fbe3ba186c684319918b631b95983caa. Latest consensus state: current_term: 1 leader_uuid: "fbe3ba186c684319918b631b95983caa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbe3ba186c684319918b631b95983caa" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 40057 } } }
I20260807 08:51:30.830029 2191 sys_catalog.cc:458] T 00000000000000000000000000000000 P fbe3ba186c684319918b631b95983caa [sys.catalog]: This master's current role is: LEADER
I20260807 08:51:30.830235 2215 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:51:30.830331 2192 sys_catalog.cc:455] T 00000000000000000000000000000000 P fbe3ba186c684319918b631b95983caa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fbe3ba186c684319918b631b95983caa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbe3ba186c684319918b631b95983caa" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 40057 } } }
I20260807 08:51:30.830415 2192 sys_catalog.cc:458] T 00000000000000000000000000000000 P fbe3ba186c684319918b631b95983caa [sys.catalog]: This master's current role is: LEADER
I20260807 08:51:30.830739 2215 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:51:30.831379 2189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:30.825860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32850 (local address 127.4.159.190:40057)
0807 08:51:30.826029 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:30.826036 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:30.826060 (+ 24us) server_negotiation.cc:407] Connection header received
0807 08:51:30.826524 (+ 464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:30.826530 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:30.826586 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:30.826689 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:51:30.826910 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:30.828357 (+ 1447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:30.829329 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:30.829908 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:30.830019 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:30.830713 (+ 694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:51:30.830717 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:51:30.830718 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:51:30.830894 (+ 176us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:51:30.830934 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:51:30.830949 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:51:30.831014 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:30.831098 (+ 84us) server_negotiation.cc:299] Negotiation successful
0807 08:51:30.831170 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":70,"threads_started":1}
I20260807 08:51:30.832100 2215 catalog_manager.cc:1347] Generated new cluster ID: ac77ca78a9bb43cc836b39c09a3fc809
I20260807 08:51:30.832151 2215 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:51:30.864670 2215 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:51:30.865185 2215 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:51:30.874099 2215 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fbe3ba186c684319918b631b95983caa: Generated new TSK 0
I20260807 08:51:30.874277 2215 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:51:30.878003 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786092453486433-4734-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786092453486433-4734-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786092453486433-4734-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786092453486433-4734-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.159.129:0
--local_ip_for_outbound_sockets=127.4.159.129
--webserver_interface=127.4.159.129
--webserver_port=0
--tserver_master_addrs=127.4.159.190:40057
--builtin_ntp_servers=127.4.159.148:33687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 08:51:30.971122 2222 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:51:30.971309 2222 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:51:30.971338 2222 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:51:30.972420 2222 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:51:30.972457 2222 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:51:30.972489 2222 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.159.129
I20260807 08:51:30.973837 2222 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:33687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786092453486433-4734-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786092453486433-4734-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.159.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786092453486433-4734-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.159.129
--webserver_port=0
--tserver_master_addrs=127.4.159.190:40057
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2222
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.159.129
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786092453486433-4734-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:51:30.974061 2222 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:51:30.974328 2222 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:51:30.978781 2229 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:51:30.978755 2231 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:51:30.978775 2228 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:51:30.978914 2222 server_base.cc:1034] running on GCE node
I20260807 08:51:30.979172 2222 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:51:30.979352 2222 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:51:30.980475 2222 hybrid_clock.cc:648] HybridClock initialized: now 1786092690980468 us; error 25 us; skew 500 ppm
I20260807 08:51:30.981536 2222 webserver.cc:460] Webserver started at http://127.4.159.129:44065/ using document root <none> and password file <none>
I20260807 08:51:30.981746 2222 fs_manager.cc:359] Metadata directory not provided
I20260807 08:51:30.981797 2222 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:51:30.981916 2222 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:51:30.982887 2222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786092453486433-4734-0/minicluster-data/ts-0/data/instance:
uuid: "6b475834c8524bf2809cac70012b76b2"
format_stamp: "Formatted at 2026-08-07 08:51:30 on dist-test-slave-6zbq"
server_key: "99ec6767836a6bb84344c36c811d50d2"
server_key_iv: "d9fd304cff48690c4a00687ab5d8273c"
server_key_version: "encryptionkey@0"
I20260807 08:51:30.983183 2222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786092453486433-4734-0/minicluster-data/ts-0/wal/instance:
uuid: "6b475834c8524bf2809cac70012b76b2"
format_stamp: "Formatted at 2026-08-07 08:51:30 on dist-test-slave-6zbq"
server_key: "99ec6767836a6bb84344c36c811d50d2"
server_key_iv: "d9fd304cff48690c4a00687ab5d8273c"
server_key_version: "encryptionkey@0"
I20260807 08:51:30.984357 2222 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 08:51:30.984973 2242 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:51:30.985112 2222 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 08:51:30.985193 2222 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786092453486433-4734-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786092453486433-4734-0/minicluster-data/ts-0/wal
uuid: "6b475834c8524bf2809cac70012b76b2"
format_stamp: "Formatted at 2026-08-07 08:51:30 on dist-test-slave-6zbq"
server_key: "99ec6767836a6bb84344c36c811d50d2"
server_key_iv: "d9fd304cff48690c4a00687ab5d8273c"
server_key_version: "encryptionkey@0"
I20260807 08:51:30.985250 2222 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786092453486433-4734-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786092453486433-4734-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786092453486433-4734-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:51:30.992563 2222 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:51:30.992792 2222 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:51:30.992902 2222 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:51:30.993088 2222 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:51:30.993357 2222 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:51:30.993389 2222 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:51:30.993413 2222 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:51:30.993430 2222 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:51:30.998704 2222 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.129:39547
I20260807 08:51:30.998742 2415 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.129:39547 every 8 connection(s)
I20260807 08:51:30.999013 2222 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786092453486433-4734-0/minicluster-data/ts-0/data/info.pb
I20260807 08:51:31.003619 2189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:30.999625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.129:58973 (local address 127.4.159.190:40057)
0807 08:51:30.999678 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:30.999680 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:30.999873 (+ 193us) server_negotiation.cc:407] Connection header received
0807 08:51:31.000302 (+ 429us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:31.000303 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:31.000315 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:31.000364 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:51:31.000985 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:31.001490 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:31.002427 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:31.002627 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:31.002683 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:31.003357 (+ 674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:51:31.003358 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:51:31.003359 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:51:31.003363 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:51:31.003378 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:51:31.003382 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:51:31.003426 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:31.003515 (+ 89us) server_negotiation.cc:299] Negotiation successful
0807 08:51:31.003543 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260807 08:51:31.003636 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 2222
I20260807 08:51:31.003720 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786092453486433-4734-0/minicluster-data/ts-0/wal/instance
I20260807 08:51:31.003788 2418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:30.999603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:40057 (local address 127.4.159.129:58973)
0807 08:51:30.999777 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:30.999805 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0807 08:51:31.000214 (+ 409us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:51:31.000426 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:51:31.000429 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:51:31.000439 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:51:31.000902 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:31.000906 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:31.001570 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:31.001571 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:31.002310 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:31.002312 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:31.002701 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:31.002702 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:31.002823 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:31.003232 (+ 409us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 08:51:31.003232 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:51:31.003261 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 08:51:31.003282 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:51:31.003433 (+ 151us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:51:31.003438 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:51:31.003440 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 08:51:31.003498 (+ 58us) client_negotiation.cc:240] Negotiation successful
0807 08:51:31.003590 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":74,"threads_started":1}
I20260807 08:51:31.003916 4734 external_mini_cluster.cc:1428] Setting key b3c64d4da9404192696ee946ab377af8
I20260807 08:51:31.004192 2416 heartbeater.cc:347] Connected to a master server at 127.4.159.190:40057
I20260807 08:51:31.004289 2416 heartbeater.cc:464] Registering TS with master...
I20260807 08:51:31.004465 2416 heartbeater.cc:511] Master 127.4.159.190:40057 requested a full tablet report, sending...
I20260807 08:51:31.004746 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786092453486433-4734-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786092453486433-4734-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786092453486433-4734-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786092453486433-4734-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.159.130:0
--local_ip_for_outbound_sockets=127.4.159.130
--webserver_interface=127.4.159.130
--webserver_port=0
--tserver_master_addrs=127.4.159.190:40057
--builtin_ntp_servers=127.4.159.148:33687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 08:51:31.004948 2127 ts_manager.cc:194] Registered new tserver with Master: 6b475834c8524bf2809cac70012b76b2 (127.4.159.129:39547)
I20260807 08:51:31.005515 2127 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.159.129:58973
W20260807 08:51:31.082768 2423 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:51:31.082938 2423 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:51:31.082957 2423 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:51:31.083994 2423 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:51:31.084025 2423 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:51:31.084048 2423 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.159.130
I20260807 08:51:31.085282 2423 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:33687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786092453486433-4734-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786092453486433-4734-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.159.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786092453486433-4734-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.159.130
--webserver_port=0
--tserver_master_addrs=127.4.159.190:40057
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2423
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.159.130
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786092453486433-4734-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:51:31.085444 2423 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:51:31.085657 2423 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:51:31.090197 2432 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:51:31.090214 2429 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:51:31.090258 2430 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:51:31.090317 2423 server_base.cc:1034] running on GCE node
I20260807 08:51:31.090570 2423 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:51:31.090771 2423 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:51:31.091904 2423 hybrid_clock.cc:648] HybridClock initialized: now 1786092691091897 us; error 25 us; skew 500 ppm
I20260807 08:51:31.092973 2423 webserver.cc:460] Webserver started at http://127.4.159.130:39461/ using document root <none> and password file <none>
I20260807 08:51:31.093166 2423 fs_manager.cc:359] Metadata directory not provided
I20260807 08:51:31.093209 2423 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:51:31.093321 2423 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:51:31.094265 2423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786092453486433-4734-0/minicluster-data/ts-1/data/instance:
uuid: "2e6cc2e71ed645b5833cc206cc210fa2"
format_stamp: "Formatted at 2026-08-07 08:51:31 on dist-test-slave-6zbq"
server_key: "ebb7f82c30e7827659205be865c5c324"
server_key_iv: "ad67c03f29557ba804bbb61d99339fb1"
server_key_version: "encryptionkey@0"
I20260807 08:51:31.094610 2423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786092453486433-4734-0/minicluster-data/ts-1/wal/instance:
uuid: "2e6cc2e71ed645b5833cc206cc210fa2"
format_stamp: "Formatted at 2026-08-07 08:51:31 on dist-test-slave-6zbq"
server_key: "ebb7f82c30e7827659205be865c5c324"
server_key_iv: "ad67c03f29557ba804bbb61d99339fb1"
server_key_version: "encryptionkey@0"
I20260807 08:51:31.095752 2423 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:51:31.096344 2441 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:51:31.096460 2423 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 08:51:31.096534 2423 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786092453486433-4734-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786092453486433-4734-0/minicluster-data/ts-1/wal
uuid: "2e6cc2e71ed645b5833cc206cc210fa2"
format_stamp: "Formatted at 2026-08-07 08:51:31 on dist-test-slave-6zbq"
server_key: "ebb7f82c30e7827659205be865c5c324"
server_key_iv: "ad67c03f29557ba804bbb61d99339fb1"
server_key_version: "encryptionkey@0"
I20260807 08:51:31.096597 2423 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786092453486433-4734-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786092453486433-4734-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786092453486433-4734-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:51:31.126199 2423 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:51:31.126444 2423 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:51:31.126559 2423 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:51:31.126749 2423 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:51:31.127022 2423 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:51:31.127054 2423 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:51:31.127074 2423 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:51:31.127095 2423 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:51:31.132328 2423 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.130:40493
I20260807 08:51:31.132385 2618 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.130:40493 every 8 connection(s)
I20260807 08:51:31.132644 2423 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786092453486433-4734-0/minicluster-data/ts-1/data/info.pb
I20260807 08:51:31.136886 2189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:31.133214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.130:36663 (local address 127.4.159.190:40057)
0807 08:51:31.133257 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:31.133259 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:31.133460 (+ 201us) server_negotiation.cc:407] Connection header received
0807 08:51:31.133854 (+ 394us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:31.133855 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:31.133863 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:31.133912 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:51:31.134541 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:31.135011 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:31.135864 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:31.136036 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:31.136077 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:31.136685 (+ 608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:51:31.136686 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:51:31.136687 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:51:31.136691 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:51:31.136704 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:51:31.136708 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:51:31.136753 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:31.136807 (+ 54us) server_negotiation.cc:299] Negotiation successful
0807 08:51:31.136828 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 08:51:31.137035 2621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:31.133183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:40057 (local address 127.4.159.130:36663)
0807 08:51:31.133353 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:31.133388 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0807 08:51:31.133784 (+ 396us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:51:31.133966 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:51:31.133969 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:51:31.133977 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:51:31.134478 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:31.134483 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:31.135067 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:31.135068 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:31.135802 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:31.135804 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:31.136086 (+ 282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:31.136087 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:31.136193 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:31.136569 (+ 376us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 08:51:31.136569 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:51:31.136593 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 08:51:31.136613 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:51:31.136753 (+ 140us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:51:31.136755 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:51:31.136758 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 08:51:31.136801 (+ 43us) client_negotiation.cc:240] Negotiation successful
0807 08:51:31.136871 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":70,"threads_started":1}
I20260807 08:51:31.137457 2619 heartbeater.cc:347] Connected to a master server at 127.4.159.190:40057
I20260807 08:51:31.137538 2619 heartbeater.cc:464] Registering TS with master...
I20260807 08:51:31.137698 2619 heartbeater.cc:511] Master 127.4.159.190:40057 requested a full tablet report, sending...
I20260807 08:51:31.138033 2127 ts_manager.cc:194] Registered new tserver with Master: 2e6cc2e71ed645b5833cc206cc210fa2 (127.4.159.130:40493)
I20260807 08:51:31.138415 2127 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.159.130:36663
I20260807 08:51:31.139679 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 2423
I20260807 08:51:31.139760 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786092453486433-4734-0/minicluster-data/ts-1/wal/instance
I20260807 08:51:31.139936 4734 external_mini_cluster.cc:1428] Setting key c19dd2061acda85c730a71c24fefe90e
I20260807 08:51:31.140714 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786092453486433-4734-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786092453486433-4734-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786092453486433-4734-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786092453486433-4734-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.159.131:0
--local_ip_for_outbound_sockets=127.4.159.131
--webserver_interface=127.4.159.131
--webserver_port=0
--tserver_master_addrs=127.4.159.190:40057
--builtin_ntp_servers=127.4.159.148:33687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 08:51:31.220103 2624 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:51:31.220269 2624 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:51:31.220288 2624 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:51:31.221374 2624 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:51:31.221411 2624 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:51:31.221437 2624 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.159.131
I20260807 08:51:31.222764 2624 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:33687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786092453486433-4734-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786092453486433-4734-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.159.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786092453486433-4734-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.159.131
--webserver_port=0
--tserver_master_addrs=127.4.159.190:40057
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2624
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.159.131
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786092453486433-4734-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:51:31.222954 2624 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:51:31.223202 2624 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:51:31.227643 2634 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:51:31.227635 2633 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:51:31.227634 2637 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:51:31.227854 2624 server_base.cc:1034] running on GCE node
I20260807 08:51:31.228093 2624 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:51:31.228317 2624 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:51:31.229453 2624 hybrid_clock.cc:648] HybridClock initialized: now 1786092691229439 us; error 27 us; skew 500 ppm
I20260807 08:51:31.230595 2624 webserver.cc:460] Webserver started at http://127.4.159.131:34689/ using document root <none> and password file <none>
I20260807 08:51:31.230815 2624 fs_manager.cc:359] Metadata directory not provided
I20260807 08:51:31.230861 2624 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:51:31.230978 2624 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:51:31.231927 2624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786092453486433-4734-0/minicluster-data/ts-2/data/instance:
uuid: "d6dae4674c8d45b3b0e25a8b4ffb0966"
format_stamp: "Formatted at 2026-08-07 08:51:31 on dist-test-slave-6zbq"
server_key: "fbb0f0bbab4fa1e87860c61c75a3ef36"
server_key_iv: "47f875c699d086f5865bdd364d62983d"
server_key_version: "encryptionkey@0"
I20260807 08:51:31.232230 2624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786092453486433-4734-0/minicluster-data/ts-2/wal/instance:
uuid: "d6dae4674c8d45b3b0e25a8b4ffb0966"
format_stamp: "Formatted at 2026-08-07 08:51:31 on dist-test-slave-6zbq"
server_key: "fbb0f0bbab4fa1e87860c61c75a3ef36"
server_key_iv: "47f875c699d086f5865bdd364d62983d"
server_key_version: "encryptionkey@0"
I20260807 08:51:31.233335 2624 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 08:51:31.233914 2645 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:51:31.234054 2624 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 08:51:31.234129 2624 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786092453486433-4734-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786092453486433-4734-0/minicluster-data/ts-2/wal
uuid: "d6dae4674c8d45b3b0e25a8b4ffb0966"
format_stamp: "Formatted at 2026-08-07 08:51:31 on dist-test-slave-6zbq"
server_key: "fbb0f0bbab4fa1e87860c61c75a3ef36"
server_key_iv: "47f875c699d086f5865bdd364d62983d"
server_key_version: "encryptionkey@0"
I20260807 08:51:31.234190 2624 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786092453486433-4734-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786092453486433-4734-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786092453486433-4734-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:51:31.261165 2624 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:51:31.261390 2624 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:51:31.261503 2624 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:51:31.261690 2624 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:51:31.261966 2624 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:51:31.261998 2624 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:51:31.262024 2624 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:51:31.262042 2624 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:51:31.267135 2624 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.131:35989
I20260807 08:51:31.267169 2815 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.131:35989 every 8 connection(s)
I20260807 08:51:31.267431 2624 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786092453486433-4734-0/minicluster-data/ts-2/data/info.pb
I20260807 08:51:31.271915 2189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:31.268029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.131:46877 (local address 127.4.159.190:40057)
0807 08:51:31.268070 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:31.268072 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:31.268250 (+ 178us) server_negotiation.cc:407] Connection header received
0807 08:51:31.268688 (+ 438us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:31.268690 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:31.268703 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:31.268766 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:51:31.269375 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:31.269856 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:31.270789 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:31.270977 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:31.271037 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:31.271666 (+ 629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:51:31.271668 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:51:31.271668 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:51:31.271674 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:51:31.271690 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:51:31.271694 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:51:31.271745 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:31.271810 (+ 65us) server_negotiation.cc:299] Negotiation successful
0807 08:51:31.271839 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260807 08:51:31.272094 2821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:31.268005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:40057 (local address 127.4.159.131:46877)
0807 08:51:31.268151 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:31.268180 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0807 08:51:31.268600 (+ 420us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:51:31.268830 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:51:31.268834 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:51:31.268844 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:51:31.269296 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:31.269300 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:31.269931 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:31.269932 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:31.270711 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:31.270713 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:31.271049 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:31.271049 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:31.271165 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:31.271550 (+ 385us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 08:51:31.271550 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:51:31.271576 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 08:51:31.271596 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:51:31.271744 (+ 148us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:51:31.271749 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:51:31.271752 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 08:51:31.271808 (+ 56us) client_negotiation.cc:240] Negotiation successful
0807 08:51:31.271902 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":60,"threads_started":1}
I20260807 08:51:31.272512 2818 heartbeater.cc:347] Connected to a master server at 127.4.159.190:40057
I20260807 08:51:31.272584 2818 heartbeater.cc:464] Registering TS with master...
I20260807 08:51:31.272754 2818 heartbeater.cc:511] Master 127.4.159.190:40057 requested a full tablet report, sending...
I20260807 08:51:31.273080 2127 ts_manager.cc:194] Registered new tserver with Master: d6dae4674c8d45b3b0e25a8b4ffb0966 (127.4.159.131:35989)
I20260807 08:51:31.273432 2127 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.159.131:46877
I20260807 08:51:31.276180 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 2624
I20260807 08:51:31.276268 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786092453486433-4734-0/minicluster-data/ts-2/wal/instance
I20260807 08:51:31.276443 4734 external_mini_cluster.cc:1428] Setting key d19ada9181658bc2524aec365f89c51c
I20260807 08:51:31.277311 4734 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 08:51:31.281026 2189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:31.278534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32856 (local address 127.4.159.190:40057)
0807 08:51:31.278567 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:31.278569 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:31.278694 (+ 125us) server_negotiation.cc:407] Connection header received
0807 08:51:31.278749 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:31.278749 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:31.278758 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:31.278802 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:51:31.278950 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:31.279383 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:31.280096 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:31.280271 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:31.280320 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:31.280825 (+ 505us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:51:31.280826 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:51:31.280826 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:51:31.280830 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:51:31.280842 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:51:31.280846 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:51:31.280888 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:31.280945 (+ 57us) server_negotiation.cc:299] Negotiation successful
0807 08:51:31.280962 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260807 08:51:31.282321 2127 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:32856:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 08:51:31.283939 2127 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:51:31.289614 2831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:31.285746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.131:35989 (local address 127.0.0.1:53106)
0807 08:51:31.285906 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:31.285942 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 08:51:31.286000 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:51:31.286327 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:51:31.286330 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:51:31.286333 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:51:31.286419 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:31.286420 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:31.287448 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:31.287449 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:31.288598 (+ 1149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:31.288601 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:31.289359 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:31.289360 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:31.289432 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:31.289444 (+ 12us) client_negotiation.cc:769] Sending connection context
0807 08:51:31.289503 (+ 59us) client_negotiation.cc:240] Negotiation successful
0807 08:51:31.289549 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"mutex_wait_us":2,"thread_start_us":57,"threads_started":1}
I20260807 08:51:31.289620 2838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:31.286044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53106 (local address 127.4.159.131:35989)
0807 08:51:31.286164 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:31.286167 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:31.286176 (+ 9us) server_negotiation.cc:407] Connection header received
0807 08:51:31.286206 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:31.286208 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:31.286221 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:31.286484 (+ 263us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:51:31.286523 (+ 39us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:31.287388 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:31.288695 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:31.289282 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:31.289339 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:31.289416 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:31.289514 (+ 98us) server_negotiation.cc:299] Negotiation successful
0807 08:51:31.289546 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89,"thread_start_us":61,"threads_started":1}
I20260807 08:51:31.289952 2832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:31.285781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.129:39547 (local address 127.0.0.1:38864)
0807 08:51:31.285911 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:31.285942 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0807 08:51:31.286000 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:51:31.286623 (+ 623us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:51:31.286624 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:51:31.286625 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:51:31.286653 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:31.286654 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:31.287941 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:31.287941 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:31.288803 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:31.288805 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:31.289715 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:31.289716 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:31.289791 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:31.289795 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 08:51:31.289849 (+ 54us) client_negotiation.cc:240] Negotiation successful
0807 08:51:31.289885 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"mutex_wait_us":26,"thread_start_us":80,"threads_started":1}
I20260807 08:51:31.290089 2836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:31.285893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38864 (local address 127.4.159.129:39547)
0807 08:51:31.286400 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:31.286404 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:31.286413 (+ 9us) server_negotiation.cc:407] Connection header received
0807 08:51:31.286452 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:31.286454 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:31.286468 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:31.286556 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:51:31.286721 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:31.287847 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:31.288876 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:31.289536 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:31.289600 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:31.289678 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:31.289884 (+ 206us) server_negotiation.cc:299] Negotiation successful
0807 08:51:31.289928 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":467,"thread_start_us":79,"threads_started":1}
I20260807 08:51:31.290331 2837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:31.285929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.130:40493 (local address 127.0.0.1:43326)
0807 08:51:31.286224 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:31.286232 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0807 08:51:31.286280 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:51:31.286574 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:51:31.286575 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:51:31.286577 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:51:31.286617 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:31.286618 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:31.287915 (+ 1297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:31.287916 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:31.289022 (+ 1106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:31.289025 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:31.290140 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:31.290141 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:31.290198 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:31.290201 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 08:51:31.290239 (+ 38us) client_negotiation.cc:240] Negotiation successful
0807 08:51:31.290274 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"mutex_wait_us":25,"thread_start_us":67,"threads_started":1}
I20260807 08:51:31.290400 2839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:31.286241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43326 (local address 127.4.159.130:40493)
0807 08:51:31.286416 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:31.286420 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:31.286428 (+ 8us) server_negotiation.cc:407] Connection header received
0807 08:51:31.286461 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:31.286463 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:31.286476 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:31.286685 (+ 209us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:51:31.286753 (+ 68us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:31.287848 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:31.289107 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:31.289962 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:31.290137 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:31.290226 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:31.290263 (+ 37us) server_negotiation.cc:299] Negotiation successful
0807 08:51:31.290300 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":87,"threads_started":1}
I20260807 08:51:31.290393 2713 tablet_service.cc:1467] Processing CreateTablet for tablet 667f3b052d534abe94f5c26acac97865 (DEFAULT_TABLE table=test-table [id=e8e101eb0a8c4c818f14ab1993d87025]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:51:31.290719 2713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 667f3b052d534abe94f5c26acac97865. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:51:31.290831 2316 tablet_service.cc:1467] Processing CreateTablet for tablet 667f3b052d534abe94f5c26acac97865 (DEFAULT_TABLE table=test-table [id=e8e101eb0a8c4c818f14ab1993d87025]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:51:31.291013 2513 tablet_service.cc:1467] Processing CreateTablet for tablet 667f3b052d534abe94f5c26acac97865 (DEFAULT_TABLE table=test-table [id=e8e101eb0a8c4c818f14ab1993d87025]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:51:31.291123 2316 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 667f3b052d534abe94f5c26acac97865. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:51:31.291258 2513 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 667f3b052d534abe94f5c26acac97865. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:51:31.293738 2841 tablet_bootstrap.cc:492] T 667f3b052d534abe94f5c26acac97865 P 2e6cc2e71ed645b5833cc206cc210fa2: Bootstrap starting.
I20260807 08:51:31.293812 2842 tablet_bootstrap.cc:492] T 667f3b052d534abe94f5c26acac97865 P d6dae4674c8d45b3b0e25a8b4ffb0966: Bootstrap starting.
I20260807 08:51:31.294122 2843 tablet_bootstrap.cc:492] T 667f3b052d534abe94f5c26acac97865 P 6b475834c8524bf2809cac70012b76b2: Bootstrap starting.
I20260807 08:51:31.294334 2841 tablet_bootstrap.cc:654] T 667f3b052d534abe94f5c26acac97865 P 2e6cc2e71ed645b5833cc206cc210fa2: Neither blocks nor log segments found. Creating new log.
I20260807 08:51:31.294417 2842 tablet_bootstrap.cc:654] T 667f3b052d534abe94f5c26acac97865 P d6dae4674c8d45b3b0e25a8b4ffb0966: Neither blocks nor log segments found. Creating new log.
I20260807 08:51:31.294661 2841 log.cc:824] T 667f3b052d534abe94f5c26acac97865 P 2e6cc2e71ed645b5833cc206cc210fa2: Log is configured to *not* fsync() on all Append() calls
I20260807 08:51:31.294695 2842 log.cc:824] T 667f3b052d534abe94f5c26acac97865 P d6dae4674c8d45b3b0e25a8b4ffb0966: Log is configured to *not* fsync() on all Append() calls
I20260807 08:51:31.294715 2843 tablet_bootstrap.cc:654] T 667f3b052d534abe94f5c26acac97865 P 6b475834c8524bf2809cac70012b76b2: Neither blocks nor log segments found. Creating new log.
I20260807 08:51:31.294993 2843 log.cc:824] T 667f3b052d534abe94f5c26acac97865 P 6b475834c8524bf2809cac70012b76b2: Log is configured to *not* fsync() on all Append() calls
I20260807 08:51:31.295436 2841 tablet_bootstrap.cc:492] T 667f3b052d534abe94f5c26acac97865 P 2e6cc2e71ed645b5833cc206cc210fa2: No bootstrap required, opened a new log
I20260807 08:51:31.295509 2841 ts_tablet_manager.cc:1397] T 667f3b052d534abe94f5c26acac97865 P 2e6cc2e71ed645b5833cc206cc210fa2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 08:51:31.295538 2842 tablet_bootstrap.cc:492] T 667f3b052d534abe94f5c26acac97865 P d6dae4674c8d45b3b0e25a8b4ffb0966: No bootstrap required, opened a new log
I20260807 08:51:31.295595 2842 ts_tablet_manager.cc:1397] T 667f3b052d534abe94f5c26acac97865 P d6dae4674c8d45b3b0e25a8b4ffb0966: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260807 08:51:31.295648 2843 tablet_bootstrap.cc:492] T 667f3b052d534abe94f5c26acac97865 P 6b475834c8524bf2809cac70012b76b2: No bootstrap required, opened a new log
I20260807 08:51:31.295713 2843 ts_tablet_manager.cc:1397] T 667f3b052d534abe94f5c26acac97865 P 6b475834c8524bf2809cac70012b76b2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 08:51:31.297567 2843 raft_consensus.cc:348] T 667f3b052d534abe94f5c26acac97865 P 6b475834c8524bf2809cac70012b76b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6dae4674c8d45b3b0e25a8b4ffb0966" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 35989 } } peers { permanent_uuid: "2e6cc2e71ed645b5833cc206cc210fa2" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 40493 } } peers { permanent_uuid: "6b475834c8524bf2809cac70012b76b2" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 39547 } }
I20260807 08:51:31.297689 2843 raft_consensus.cc:374] T 667f3b052d534abe94f5c26acac97865 P 6b475834c8524bf2809cac70012b76b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:51:31.297720 2843 raft_consensus.cc:729] T 667f3b052d534abe94f5c26acac97865 P 6b475834c8524bf2809cac70012b76b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b475834c8524bf2809cac70012b76b2, State: Initialized, Role: FOLLOWER
I20260807 08:51:31.297855 2843 consensus_queue.cc:260] T 667f3b052d534abe94f5c26acac97865 P 6b475834c8524bf2809cac70012b76b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6dae4674c8d45b3b0e25a8b4ffb0966" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 35989 } } peers { permanent_uuid: "2e6cc2e71ed645b5833cc206cc210fa2" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 40493 } } peers { permanent_uuid: "6b475834c8524bf2809cac70012b76b2" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 39547 } }
I20260807 08:51:31.297905 2841 raft_consensus.cc:348] T 667f3b052d534abe94f5c26acac97865 P 2e6cc2e71ed645b5833cc206cc210fa2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6dae4674c8d45b3b0e25a8b4ffb0966" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 35989 } } peers { permanent_uuid: "2e6cc2e71ed645b5833cc206cc210fa2" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 40493 } } peers { permanent_uuid: "6b475834c8524bf2809cac70012b76b2" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 39547 } }
I20260807 08:51:31.297905 2842 raft_consensus.cc:348] T 667f3b052d534abe94f5c26acac97865 P d6dae4674c8d45b3b0e25a8b4ffb0966 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6dae4674c8d45b3b0e25a8b4ffb0966" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 35989 } } peers { permanent_uuid: "2e6cc2e71ed645b5833cc206cc210fa2" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 40493 } } peers { permanent_uuid: "6b475834c8524bf2809cac70012b76b2" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 39547 } }
I20260807 08:51:31.298056 2841 raft_consensus.cc:374] T 667f3b052d534abe94f5c26acac97865 P 2e6cc2e71ed645b5833cc206cc210fa2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:51:31.298056 2842 raft_consensus.cc:374] T 667f3b052d534abe94f5c26acac97865 P d6dae4674c8d45b3b0e25a8b4ffb0966 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:51:31.298082 2843 ts_tablet_manager.cc:1428] T 667f3b052d534abe94f5c26acac97865 P 6b475834c8524bf2809cac70012b76b2: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260807 08:51:31.298099 2841 raft_consensus.cc:729] T 667f3b052d534abe94f5c26acac97865 P 2e6cc2e71ed645b5833cc206cc210fa2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e6cc2e71ed645b5833cc206cc210fa2, State: Initialized, Role: FOLLOWER
I20260807 08:51:31.298100 2842 raft_consensus.cc:729] T 667f3b052d534abe94f5c26acac97865 P d6dae4674c8d45b3b0e25a8b4ffb0966 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6dae4674c8d45b3b0e25a8b4ffb0966, State: Initialized, Role: FOLLOWER
I20260807 08:51:31.298187 2416 heartbeater.cc:503] Master 127.4.159.190:40057 was elected leader, sending a full tablet report...
I20260807 08:51:31.298208 2841 consensus_queue.cc:260] T 667f3b052d534abe94f5c26acac97865 P 2e6cc2e71ed645b5833cc206cc210fa2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6dae4674c8d45b3b0e25a8b4ffb0966" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 35989 } } peers { permanent_uuid: "2e6cc2e71ed645b5833cc206cc210fa2" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 40493 } } peers { permanent_uuid: "6b475834c8524bf2809cac70012b76b2" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 39547 } }
I20260807 08:51:31.298210 2842 consensus_queue.cc:260] T 667f3b052d534abe94f5c26acac97865 P d6dae4674c8d45b3b0e25a8b4ffb0966 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6dae4674c8d45b3b0e25a8b4ffb0966" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 35989 } } peers { permanent_uuid: "2e6cc2e71ed645b5833cc206cc210fa2" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 40493 } } peers { permanent_uuid: "6b475834c8524bf2809cac70012b76b2" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 39547 } }
I20260807 08:51:31.298470 2842 ts_tablet_manager.cc:1428] T 667f3b052d534abe94f5c26acac97865 P d6dae4674c8d45b3b0e25a8b4ffb0966: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260807 08:51:31.298470 2841 ts_tablet_manager.cc:1428] T 667f3b052d534abe94f5c26acac97865 P 2e6cc2e71ed645b5833cc206cc210fa2: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260807 08:51:31.298575 2619 heartbeater.cc:503] Master 127.4.159.190:40057 was elected leader, sending a full tablet report...
I20260807 08:51:31.298808 2818 heartbeater.cc:503] Master 127.4.159.190:40057 was elected leader, sending a full tablet report...
W20260807 08:51:31.383086 2620 tablet.cc:2369] T 667f3b052d534abe94f5c26acac97865 P 2e6cc2e71ed645b5833cc206cc210fa2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 08:51:31.499585 2417 tablet.cc:2369] T 667f3b052d534abe94f5c26acac97865 P 6b475834c8524bf2809cac70012b76b2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 08:51:31.517875 2820 tablet.cc:2369] T 667f3b052d534abe94f5c26acac97865 P d6dae4674c8d45b3b0e25a8b4ffb0966: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 08:51:31.550238 2854 raft_consensus.cc:482] T 667f3b052d534abe94f5c26acac97865 P d6dae4674c8d45b3b0e25a8b4ffb0966 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:51:31.550365 2854 raft_consensus.cc:504] T 667f3b052d534abe94f5c26acac97865 P d6dae4674c8d45b3b0e25a8b4ffb0966 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6dae4674c8d45b3b0e25a8b4ffb0966" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 35989 } } peers { permanent_uuid: "2e6cc2e71ed645b5833cc206cc210fa2" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 40493 } } peers { permanent_uuid: "6b475834c8524bf2809cac70012b76b2" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 39547 } }
I20260807 08:51:31.550706 2854 leader_election.cc:288] T 667f3b052d534abe94f5c26acac97865 P d6dae4674c8d45b3b0e25a8b4ffb0966 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2e6cc2e71ed645b5833cc206cc210fa2 (127.4.159.130:40493), 6b475834c8524bf2809cac70012b76b2 (127.4.159.129:39547)
I20260807 08:51:31.553201 2821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:31.550849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.130:40493 (local address 127.4.159.131:56983)
0807 08:51:31.550914 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:31.550926 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 08:51:31.550969 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:51:31.551100 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:51:31.551100 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:51:31.551102 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:51:31.551165 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:31.551167 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:31.551774 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:31.551774 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:31.552481 (+ 707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:31.552482 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:31.552983 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:31.552984 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:31.553044 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:31.553047 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 08:51:31.553091 (+ 44us) client_negotiation.cc:240] Negotiation successful
0807 08:51:31.553131 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43,"mutex_wait_us":1}
I20260807 08:51:31.553201 2839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:31.550901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.131:56983 (local address 127.4.159.130:40493)
0807 08:51:31.550942 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:31.550944 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:31.550954 (+ 10us) server_negotiation.cc:407] Connection header received
0807 08:51:31.551033 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:31.551034 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:31.551045 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:31.551089 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:51:31.551226 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:31.551696 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:31.552569 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:31.552912 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:31.552972 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:31.552981 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:31.553103 (+ 122us) server_negotiation.cc:299] Negotiation successful
0807 08:51:31.553138 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 08:51:31.553350 2836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:31.550920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.159.131:60239 (local address 127.4.159.129:39547)
0807 08:51:31.550959 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:31.550961 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:31.551136 (+ 175us) server_negotiation.cc:407] Connection header received
0807 08:51:31.551194 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:31.551195 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:31.551206 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:31.551253 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:51:31.551391 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:31.551847 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:31.552711 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:31.553049 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:31.553107 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:31.553114 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:31.553258 (+ 144us) server_negotiation.cc:299] Negotiation successful
0807 08:51:31.553292 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 08:51:31.553357 2856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:31.550853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.129:39547 (local address 127.4.159.131:60239)
0807 08:51:31.551066 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:31.551075 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 08:51:31.551130 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:51:31.551261 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:51:31.551262 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:51:31.551263 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:51:31.551332 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:31.551333 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:31.551917 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:31.551918 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:31.552627 (+ 709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:31.552628 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:31.553120 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:31.553120 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:31.553197 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:31.553200 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 08:51:31.553244 (+ 44us) client_negotiation.cc:240] Negotiation successful
0807 08:51:31.553279 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"mutex_wait_us":56,"thread_start_us":97,"threads_started":1}
I20260807 08:51:31.553526 2544 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "667f3b052d534abe94f5c26acac97865" candidate_uuid: "d6dae4674c8d45b3b0e25a8b4ffb0966" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e6cc2e71ed645b5833cc206cc210fa2" is_pre_election: true
I20260807 08:51:31.553629 2353 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "667f3b052d534abe94f5c26acac97865" candidate_uuid: "d6dae4674c8d45b3b0e25a8b4ffb0966" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b475834c8524bf2809cac70012b76b2" is_pre_election: true
I20260807 08:51:31.553676 2544 raft_consensus.cc:2454] T 667f3b052d534abe94f5c26acac97865 P 2e6cc2e71ed645b5833cc206cc210fa2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6dae4674c8d45b3b0e25a8b4ffb0966 in term 0.
I20260807 08:51:31.553753 2353 raft_consensus.cc:2454] T 667f3b052d534abe94f5c26acac97865 P 6b475834c8524bf2809cac70012b76b2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6dae4674c8d45b3b0e25a8b4ffb0966 in term 0.
I20260807 08:51:31.553903 2649 leader_election.cc:302] T 667f3b052d534abe94f5c26acac97865 P d6dae4674c8d45b3b0e25a8b4ffb0966 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e6cc2e71ed645b5833cc206cc210fa2, d6dae4674c8d45b3b0e25a8b4ffb0966; no voters:
I20260807 08:51:31.554069 2854 raft_consensus.cc:2781] T 667f3b052d534abe94f5c26acac97865 P d6dae4674c8d45b3b0e25a8b4ffb0966 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:51:31.554118 2854 raft_consensus.cc:482] T 667f3b052d534abe94f5c26acac97865 P d6dae4674c8d45b3b0e25a8b4ffb0966 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:51:31.554132 2854 raft_consensus.cc:3037] T 667f3b052d534abe94f5c26acac97865 P d6dae4674c8d45b3b0e25a8b4ffb0966 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:51:31.554832 2854 raft_consensus.cc:504] T 667f3b052d534abe94f5c26acac97865 P d6dae4674c8d45b3b0e25a8b4ffb0966 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6dae4674c8d45b3b0e25a8b4ffb0966" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 35989 } } peers { permanent_uuid: "2e6cc2e71ed645b5833cc206cc210fa2" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 40493 } } peers { permanent_uuid: "6b475834c8524bf2809cac70012b76b2" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 39547 } }
I20260807 08:51:31.554947 2854 leader_election.cc:288] T 667f3b052d534abe94f5c26acac97865 P d6dae4674c8d45b3b0e25a8b4ffb0966 [CANDIDATE]: Term 1 election: Requested vote from peers 2e6cc2e71ed645b5833cc206cc210fa2 (127.4.159.130:40493), 6b475834c8524bf2809cac70012b76b2 (127.4.159.129:39547)
I20260807 08:51:31.555079 2353 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "667f3b052d534abe94f5c26acac97865" candidate_uuid: "d6dae4674c8d45b3b0e25a8b4ffb0966" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b475834c8524bf2809cac70012b76b2"
I20260807 08:51:31.555094 2544 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "667f3b052d534abe94f5c26acac97865" candidate_uuid: "d6dae4674c8d45b3b0e25a8b4ffb0966" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e6cc2e71ed645b5833cc206cc210fa2"
I20260807 08:51:31.555152 2544 raft_consensus.cc:3037] T 667f3b052d534abe94f5c26acac97865 P 2e6cc2e71ed645b5833cc206cc210fa2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:51:31.555150 2353 raft_consensus.cc:3037] T 667f3b052d534abe94f5c26acac97865 P 6b475834c8524bf2809cac70012b76b2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:51:31.555787 2544 raft_consensus.cc:2454] T 667f3b052d534abe94f5c26acac97865 P 2e6cc2e71ed645b5833cc206cc210fa2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6dae4674c8d45b3b0e25a8b4ffb0966 in term 1.
I20260807 08:51:31.555955 2649 leader_election.cc:302] T 667f3b052d534abe94f5c26acac97865 P d6dae4674c8d45b3b0e25a8b4ffb0966 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e6cc2e71ed645b5833cc206cc210fa2, d6dae4674c8d45b3b0e25a8b4ffb0966; no voters:
I20260807 08:51:31.555954 2353 raft_consensus.cc:2454] T 667f3b052d534abe94f5c26acac97865 P 6b475834c8524bf2809cac70012b76b2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6dae4674c8d45b3b0e25a8b4ffb0966 in term 1.
I20260807 08:51:31.556049 2854 raft_consensus.cc:2781] T 667f3b052d534abe94f5c26acac97865 P d6dae4674c8d45b3b0e25a8b4ffb0966 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:51:31.556191 2854 raft_consensus.cc:686] T 667f3b052d534abe94f5c26acac97865 P d6dae4674c8d45b3b0e25a8b4ffb0966 [term 1 LEADER]: Becoming Leader. State: Replica: d6dae4674c8d45b3b0e25a8b4ffb0966, State: Running, Role: LEADER
I20260807 08:51:31.556282 2854 consensus_queue.cc:237] T 667f3b052d534abe94f5c26acac97865 P d6dae4674c8d45b3b0e25a8b4ffb0966 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6dae4674c8d45b3b0e25a8b4ffb0966" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 35989 } } peers { permanent_uuid: "2e6cc2e71ed645b5833cc206cc210fa2" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 40493 } } peers { permanent_uuid: "6b475834c8524bf2809cac70012b76b2" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 39547 } }
I20260807 08:51:31.557018 2125 catalog_manager.cc:5495] T 667f3b052d534abe94f5c26acac97865 P d6dae4674c8d45b3b0e25a8b4ffb0966 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6dae4674c8d45b3b0e25a8b4ffb0966 (127.4.159.131). New cstate: current_term: 1 leader_uuid: "d6dae4674c8d45b3b0e25a8b4ffb0966" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6dae4674c8d45b3b0e25a8b4ffb0966" member_type: VOTER last_known_addr { host: "127.4.159.131" port: 35989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2e6cc2e71ed645b5833cc206cc210fa2" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 40493 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b475834c8524bf2809cac70012b76b2" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 39547 } health_report { overall_health: UNKNOWN } } }
I20260807 08:51:31.567967 2838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:31.565917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53114 (local address 127.4.159.131:35989)
0807 08:51:31.565969 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:31.565971 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:31.565993 (+ 22us) server_negotiation.cc:407] Connection header received
0807 08:51:31.566020 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:31.566021 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:31.566037 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:31.566087 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:51:31.566190 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:31.566626 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:31.567319 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:31.567499 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:31.567550 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:31.567681 (+ 131us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:51:31.567773 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:51:31.567840 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:31.567876 (+ 36us) server_negotiation.cc:299] Negotiation successful
0807 08:51:31.567892 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260807 08:51:31.569327 2353 raft_consensus.cc:1261] T 667f3b052d534abe94f5c26acac97865 P 6b475834c8524bf2809cac70012b76b2 [term 1 FOLLOWER]: Refusing update from remote peer d6dae4674c8d45b3b0e25a8b4ffb0966: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:51:31.569334 2544 raft_consensus.cc:1261] T 667f3b052d534abe94f5c26acac97865 P 2e6cc2e71ed645b5833cc206cc210fa2 [term 1 FOLLOWER]: Refusing update from remote peer d6dae4674c8d45b3b0e25a8b4ffb0966: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:51:31.569633 2860 consensus_queue.cc:1035] T 667f3b052d534abe94f5c26acac97865 P d6dae4674c8d45b3b0e25a8b4ffb0966 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b475834c8524bf2809cac70012b76b2" member_type: VOTER last_known_addr { host: "127.4.159.129" port: 39547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:51:31.569736 2854 consensus_queue.cc:1035] T 667f3b052d534abe94f5c26acac97865 P d6dae4674c8d45b3b0e25a8b4ffb0966 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2e6cc2e71ed645b5833cc206cc210fa2" member_type: VOTER last_known_addr { host: "127.4.159.130" port: 40493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:51:31.572009 2867 mvcc.cc:205] Tried to move back new op lower bound from 7315835664665993216 to 7315835664614928384. Current Snapshot: MvccSnapshot[applied={T|T < 7315835664665993216}]
I20260807 08:51:31.572734 2864 mvcc.cc:205] Tried to move back new op lower bound from 7315835664665993216 to 7315835664614928384. Current Snapshot: MvccSnapshot[applied={T|T < 7315835664665993216}]
I20260807 08:51:31.574059 2125 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:32856:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:51:31.574147 2125 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:32856:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:51:31.574797 2125 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fbe3ba186c684319918b631b95983caa: Sending DeleteTablet for 3 replicas of tablet 667f3b052d534abe94f5c26acac97865
I20260807 08:51:31.575101 2713 tablet_service.cc:1514] Processing DeleteTablet for tablet 667f3b052d534abe94f5c26acac97865 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:51:31 UTC) from {username='slave'} at 127.0.0.1:53106
I20260807 08:51:31.575101 2316 tablet_service.cc:1514] Processing DeleteTablet for tablet 667f3b052d534abe94f5c26acac97865 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:51:31 UTC) from {username='slave'} at 127.0.0.1:38864
I20260807 08:51:31.575150 2513 tablet_service.cc:1514] Processing DeleteTablet for tablet 667f3b052d534abe94f5c26acac97865 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:51:31 UTC) from {username='slave'} at 127.0.0.1:43326
I20260807 08:51:31.575335 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 2222
I20260807 08:51:31.581604 2884 tablet_replica.cc:330] T 667f3b052d534abe94f5c26acac97865 P 2e6cc2e71ed645b5833cc206cc210fa2: stopping tablet replica
I20260807 08:51:31.581722 2884 raft_consensus.cc:2229] T 667f3b052d534abe94f5c26acac97865 P 2e6cc2e71ed645b5833cc206cc210fa2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:51:31.581828 2884 raft_consensus.cc:2258] T 667f3b052d534abe94f5c26acac97865 P 2e6cc2e71ed645b5833cc206cc210fa2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:51:31.587735 2884 ts_tablet_manager.cc:1905] T 667f3b052d534abe94f5c26acac97865 P 2e6cc2e71ed645b5833cc206cc210fa2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:51:31.589283 2884 ts_tablet_manager.cc:1918] T 667f3b052d534abe94f5c26acac97865 P 2e6cc2e71ed645b5833cc206cc210fa2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:51:31.589357 2884 log.cc:1196] T 667f3b052d534abe94f5c26acac97865 P 2e6cc2e71ed645b5833cc206cc210fa2: Deleting WAL directory at /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786092453486433-4734-0/minicluster-data/ts-1/wal/wals/667f3b052d534abe94f5c26acac97865
I20260807 08:51:31.589358 2882 tablet_replica.cc:330] T 667f3b052d534abe94f5c26acac97865 P d6dae4674c8d45b3b0e25a8b4ffb0966: stopping tablet replica
I20260807 08:51:31.589458 2882 raft_consensus.cc:2229] T 667f3b052d534abe94f5c26acac97865 P d6dae4674c8d45b3b0e25a8b4ffb0966 [term 1 LEADER]: Raft consensus shutting down.
I20260807 08:51:31.589555 2882 raft_consensus.cc:2258] T 667f3b052d534abe94f5c26acac97865 P d6dae4674c8d45b3b0e25a8b4ffb0966 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:51:31.589591 2884 ts_tablet_manager.cc:1939] T 667f3b052d534abe94f5c26acac97865 P 2e6cc2e71ed645b5833cc206cc210fa2: Deleting consensus metadata
I20260807 08:51:31.590032 2882 ts_tablet_manager.cc:1905] T 667f3b052d534abe94f5c26acac97865 P d6dae4674c8d45b3b0e25a8b4ffb0966: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:51:31.590092 2102 catalog_manager.cc:4841] TS 2e6cc2e71ed645b5833cc206cc210fa2 (127.4.159.130:40493): tablet 667f3b052d534abe94f5c26acac97865 (table test-table [id=e8e101eb0a8c4c818f14ab1993d87025]) successfully deleted
I20260807 08:51:31.591583 2882 ts_tablet_manager.cc:1918] T 667f3b052d534abe94f5c26acac97865 P d6dae4674c8d45b3b0e25a8b4ffb0966: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:51:31.591703 2882 log.cc:1196] T 667f3b052d534abe94f5c26acac97865 P d6dae4674c8d45b3b0e25a8b4ffb0966: Deleting WAL directory at /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786092453486433-4734-0/minicluster-data/ts-2/wal/wals/667f3b052d534abe94f5c26acac97865
I20260807 08:51:31.591988 2882 ts_tablet_manager.cc:1939] T 667f3b052d534abe94f5c26acac97865 P d6dae4674c8d45b3b0e25a8b4ffb0966: Deleting consensus metadata
I20260807 08:51:31.592422 2102 catalog_manager.cc:4841] TS d6dae4674c8d45b3b0e25a8b4ffb0966 (127.4.159.131:35989): tablet 667f3b052d534abe94f5c26acac97865 (table test-table [id=e8e101eb0a8c4c818f14ab1993d87025]) successfully deleted
W20260807 08:51:31.594054 2106 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.159.129:39547 (error 108)
I20260807 08:51:31.594532 2837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:31.594409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.129:39547 (local address 127.0.0.1:38876)
0807 08:51:31.594446 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:31.594500 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.129:39547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
I20260807 08:51:31.594620 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 2423
W20260807 08:51:31.600493 2106 catalog_manager.cc:4584] TS 6b475834c8524bf2809cac70012b76b2 (127.4.159.129:39547): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 667f3b052d534abe94f5c26acac97865: Network error: Client connection negotiation failed: client connection to 127.4.159.129:39547: connect: Connection refused (error 111)
I20260807 08:51:31.603796 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 2624
I20260807 08:51:31.613003 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 2081
2026-08-07T08:51:31Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (929 ms)
[----------] 6 tests from EncryptionPolicyTest (9142 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786092453486433-4734-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:51:31 dist-test-slave-6zbq krb5kdc[2894](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:51:31 dist-test-slave-6zbq krb5kdc[2894](info): set up 2 sockets
Aug 07 08:51:31 dist-test-slave-6zbq krb5kdc[2894](info): commencing operation
krb5kdc: starting...
W20260807 08:51:32.727741 4734 mini_kdc.cc:121] Time spent starting KDC: real 1.094s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786092453486433-4734-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786092453486433-4734-0/krb5kdc/test-user.keytab.
Aug 07 08:51:32 dist-test-slave-6zbq krb5kdc[2894](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092692, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:51:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:51:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786092453486433-4734-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786092453486433-4734-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 08:51:32.834555 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786092453486433-4734-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786092453486433-4734-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786092453486433-4734-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43261
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786092453486433-4734-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43261
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:51:32.913589 2914 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:51:32.913770 2914 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:51:32.913792 2914 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:51:32.914863 2914 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:51:32.914896 2914 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:51:32.914909 2914 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:51:32.914920 2914 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:51:32.914937 2914 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:51:32.916301 2914 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786092453486433-4734-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786092453486433-4734-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43261
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786092453486433-4734-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43261
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2914
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786092453486433-4734-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:51:32.916474 2914 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:51:32.916632 2914 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:51:32.920959 2923 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:51:32.920985 2924 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:51:32.921039 2914 server_base.cc:1034] running on GCE node
W20260807 08:51:32.920980 2926 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:51:32.921754 2914 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:51:32.921984 2914 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:51:32.923111 2914 hybrid_clock.cc:648] HybridClock initialized: now 1786092692923099 us; error 29 us; skew 500 ppm
Aug 07 08:51:32 dist-test-slave-6zbq krb5kdc[2894](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092692, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:51:32.925325 2914 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 08:51:32.925737 2914 webserver.cc:460] Webserver started at http://127.0.0.1:44447/ using document root <none> and password file <none>
I20260807 08:51:32.925926 2914 fs_manager.cc:359] Metadata directory not provided
I20260807 08:51:32.925973 2914 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:51:32.926092 2914 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:51:32.927086 2914 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786092453486433-4734-0/minicluster-data/master-0/data/instance:
uuid: "8e908b9d1b654cc5a7ba8aeb61d16f40"
format_stamp: "Formatted at 2026-08-07 08:51:32 on dist-test-slave-6zbq"
server_key: "0ba0b8461d31cb8bb2f4d28b09e99588"
server_key_iv: "7ab8f7944870a53b2e7589ae276ed1a5"
server_key_version: "encryptionkey@0"
I20260807 08:51:32.927404 2914 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786092453486433-4734-0/minicluster-data/master-0/wal/instance:
uuid: "8e908b9d1b654cc5a7ba8aeb61d16f40"
format_stamp: "Formatted at 2026-08-07 08:51:32 on dist-test-slave-6zbq"
server_key: "0ba0b8461d31cb8bb2f4d28b09e99588"
server_key_iv: "7ab8f7944870a53b2e7589ae276ed1a5"
server_key_version: "encryptionkey@0"
I20260807 08:51:32.928575 2914 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260807 08:51:32.929212 2939 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:51:32.929353 2914 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 08:51:32.929433 2914 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786092453486433-4734-0/minicluster-data/master-0/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786092453486433-4734-0/minicluster-data/master-0/wal
uuid: "8e908b9d1b654cc5a7ba8aeb61d16f40"
format_stamp: "Formatted at 2026-08-07 08:51:32 on dist-test-slave-6zbq"
server_key: "0ba0b8461d31cb8bb2f4d28b09e99588"
server_key_iv: "7ab8f7944870a53b2e7589ae276ed1a5"
server_key_version: "encryptionkey@0"
I20260807 08:51:32.929490 2914 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786092453486433-4734-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786092453486433-4734-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786092453486433-4734-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:51:32.938457 2914 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:51:32.940160 2914 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:51:32.940282 2914 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:51:32.944167 2914 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43261
I20260807 08:51:32.944196 3016 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43261 every 8 connection(s)
I20260807 08:51:32.944492 2914 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
I20260807 08:51:32.945384 3017 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:51:32.947584 3017 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e908b9d1b654cc5a7ba8aeb61d16f40: Bootstrap starting.
I20260807 08:51:32.948119 3017 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8e908b9d1b654cc5a7ba8aeb61d16f40: Neither blocks nor log segments found. Creating new log.
I20260807 08:51:32.948408 3017 log.cc:824] T 00000000000000000000000000000000 P 8e908b9d1b654cc5a7ba8aeb61d16f40: Log is configured to *not* fsync() on all Append() calls
I20260807 08:51:32.948467 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 2914
I20260807 08:51:32.948570 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786092453486433-4734-0/minicluster-data/master-0/wal/instance
I20260807 08:51:32.948786 4734 external_mini_cluster.cc:1428] Setting key 218a926c371be1a198def8a123c3bfa2
I20260807 08:51:32.949153 3017 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e908b9d1b654cc5a7ba8aeb61d16f40: No bootstrap required, opened a new log
I20260807 08:51:32.951112 3017 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8e908b9d1b654cc5a7ba8aeb61d16f40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e908b9d1b654cc5a7ba8aeb61d16f40" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43261 } }
I20260807 08:51:32.951287 3017 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8e908b9d1b654cc5a7ba8aeb61d16f40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:51:32.951332 3017 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8e908b9d1b654cc5a7ba8aeb61d16f40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e908b9d1b654cc5a7ba8aeb61d16f40, State: Initialized, Role: FOLLOWER
I20260807 08:51:32.951458 3017 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8e908b9d1b654cc5a7ba8aeb61d16f40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e908b9d1b654cc5a7ba8aeb61d16f40" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43261 } }
I20260807 08:51:32.951534 3017 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8e908b9d1b654cc5a7ba8aeb61d16f40 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:51:32.951570 3017 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8e908b9d1b654cc5a7ba8aeb61d16f40 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:51:32.951613 3017 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8e908b9d1b654cc5a7ba8aeb61d16f40 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:51:32.952266 3017 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8e908b9d1b654cc5a7ba8aeb61d16f40 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e908b9d1b654cc5a7ba8aeb61d16f40" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43261 } }
I20260807 08:51:32.952373 3017 leader_election.cc:302] T 00000000000000000000000000000000 P 8e908b9d1b654cc5a7ba8aeb61d16f40 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8e908b9d1b654cc5a7ba8aeb61d16f40; no voters:
I20260807 08:51:32.952507 3017 leader_election.cc:288] T 00000000000000000000000000000000 P 8e908b9d1b654cc5a7ba8aeb61d16f40 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:51:32.952549 3023 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8e908b9d1b654cc5a7ba8aeb61d16f40 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:51:32.952723 3017 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8e908b9d1b654cc5a7ba8aeb61d16f40 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:51:32.952739 3023 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8e908b9d1b654cc5a7ba8aeb61d16f40 [term 1 LEADER]: Becoming Leader. State: Replica: 8e908b9d1b654cc5a7ba8aeb61d16f40, State: Running, Role: LEADER
I20260807 08:51:32.952894 3023 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8e908b9d1b654cc5a7ba8aeb61d16f40 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e908b9d1b654cc5a7ba8aeb61d16f40" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43261 } }
I20260807 08:51:32.953306 3026 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e908b9d1b654cc5a7ba8aeb61d16f40 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8e908b9d1b654cc5a7ba8aeb61d16f40. Latest consensus state: current_term: 1 leader_uuid: "8e908b9d1b654cc5a7ba8aeb61d16f40" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e908b9d1b654cc5a7ba8aeb61d16f40" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43261 } } }
I20260807 08:51:32.953387 3026 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e908b9d1b654cc5a7ba8aeb61d16f40 [sys.catalog]: This master's current role is: LEADER
I20260807 08:51:32.953548 3024 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e908b9d1b654cc5a7ba8aeb61d16f40 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8e908b9d1b654cc5a7ba8aeb61d16f40" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e908b9d1b654cc5a7ba8aeb61d16f40" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43261 } } }
I20260807 08:51:32.953589 3035 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:51:32.953600 3024 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e908b9d1b654cc5a7ba8aeb61d16f40 [sys.catalog]: This master's current role is: LEADER
I20260807 08:51:32.954186 3035 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:51:32.956463 3035 catalog_manager.cc:1347] Generated new cluster ID: 236fad2886c244298b59d70a743d8c93
I20260807 08:51:32.956511 3035 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 08:51:32 dist-test-slave-6zbq krb5kdc[2894](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092692, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 08:51:32.960333 3021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:32.949297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55181 (local address 127.0.0.1:43261)
0807 08:51:32.949470 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:32.949479 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:32.949510 (+ 31us) server_negotiation.cc:407] Connection header received
0807 08:51:32.950392 (+ 882us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:32.950401 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:32.950461 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:32.950599 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:51:32.951162 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:32.952759 (+ 1597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:32.953859 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:32.954451 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:32.954555 (+ 104us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:32.957485 (+ 2930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:51:32.957488 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:51:32.957489 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:51:32.957498 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:51:32.958963 (+ 1465us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:51:32.959311 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:51:32.959314 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:51:32.959314 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:51:32.959393 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:51:32.959594 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:51:32.959595 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:51:32.959595 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:51:32.959806 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:51:32.959941 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:32.960067 (+ 126us) server_negotiation.cc:299] Negotiation successful
0807 08:51:32.960159 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":80,"threads_started":1}
I20260807 08:51:32.961231 4734 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 08:51:32.972083 3021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:32.962623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36501 (local address 127.0.0.1:43261)
0807 08:51:32.962659 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:32.962661 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:32.966429 (+ 3768us) server_negotiation.cc:407] Connection header received
0807 08:51:32.966560 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:32.966561 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:32.966598 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:32.966627 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:51:32.967424 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:32.967904 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:32.968582 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:32.968807 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:32.968861 (+ 54us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:32.969872 (+ 1011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:51:32.969873 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:51:32.969873 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:51:32.969880 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:51:32.971085 (+ 1205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:51:32.971407 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:51:32.971408 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:51:32.971408 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:51:32.971460 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:51:32.971639 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:51:32.971639 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:51:32.971640 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:51:32.971735 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:51:32.971821 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:32.971936 (+ 115us) server_negotiation.cc:299] Negotiation successful
0807 08:51:32.971995 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 08:51:32.974299 3035 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:51:32.974979 3035 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:51:32.981328 3035 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8e908b9d1b654cc5a7ba8aeb61d16f40: Generated new TSK 0
I20260807 08:51:32.981487 3035 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:51:32.986477 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 2914
W20260807 08:51:32.999442 2889 connection.cc:535] client connection to 127.0.0.1:43261 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43261): Connection reset by peer (error 104)
2026-08-07T08:51:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1378 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786092453486433-4734-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:51:33 dist-test-slave-6zbq krb5kdc[3057](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:51:33 dist-test-slave-6zbq krb5kdc[3057](info): set up 2 sockets
Aug 07 08:51:33 dist-test-slave-6zbq krb5kdc[3057](info): commencing operation
krb5kdc: starting...
W20260807 08:51:34.117605 4734 mini_kdc.cc:121] Time spent starting KDC: real 1.105s user 0.005s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786092453486433-4734-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786092453486433-4734-0/krb5kdc/test-user.keytab.
Aug 07 08:51:34 dist-test-slave-6zbq krb5kdc[3057](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092694, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:51:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:51:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786092453486433-4734-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786092453486433-4734-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 08:51:34.224632 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786092453486433-4734-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786092453486433-4734-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786092453486433-4734-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43189
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786092453486433-4734-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43189
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:51:34.303265 3072 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:51:34.303447 3072 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:51:34.303476 3072 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:51:34.304587 3072 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:51:34.304627 3072 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:51:34.304646 3072 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:51:34.304661 3072 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:51:34.304678 3072 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:51:34.306114 3072 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786092453486433-4734-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786092453486433-4734-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43189
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786092453486433-4734-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43189
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3072
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786092453486433-4734-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:51:34.306377 3072 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:51:34.306569 3072 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:51:34.310937 3086 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:51:34.310941 3088 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:51:34.310995 3082 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:51:34.311223 3072 server_base.cc:1034] running on GCE node
I20260807 08:51:34.311446 3072 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:51:34.311756 3072 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:51:34.312914 3072 hybrid_clock.cc:648] HybridClock initialized: now 1786092694312904 us; error 27 us; skew 500 ppm
Aug 07 08:51:34 dist-test-slave-6zbq krb5kdc[3057](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092694, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:51:34.319383 3072 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 08:51:34.319815 3072 webserver.cc:460] Webserver started at http://127.0.0.1:46095/ using document root <none> and password file <none>
I20260807 08:51:34.320004 3072 fs_manager.cc:359] Metadata directory not provided
I20260807 08:51:34.320050 3072 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:51:34.320174 3072 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:51:34.321156 3072 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786092453486433-4734-0/minicluster-data/master-0/data/instance:
uuid: "8a2414f2b5704b64849238d709a16b39"
format_stamp: "Formatted at 2026-08-07 08:51:34 on dist-test-slave-6zbq"
server_key: "4c082adc4e843433dd9b517b088afc2e"
server_key_iv: "254fb064925219aefed2062d1d8bc5a1"
server_key_version: "encryptionkey@0"
I20260807 08:51:34.321470 3072 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786092453486433-4734-0/minicluster-data/master-0/wal/instance:
uuid: "8a2414f2b5704b64849238d709a16b39"
format_stamp: "Formatted at 2026-08-07 08:51:34 on dist-test-slave-6zbq"
server_key: "4c082adc4e843433dd9b517b088afc2e"
server_key_iv: "254fb064925219aefed2062d1d8bc5a1"
server_key_version: "encryptionkey@0"
I20260807 08:51:34.322670 3072 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 08:51:34.323370 3096 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:51:34.323529 3072 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 08:51:34.323611 3072 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786092453486433-4734-0/minicluster-data/master-0/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786092453486433-4734-0/minicluster-data/master-0/wal
uuid: "8a2414f2b5704b64849238d709a16b39"
format_stamp: "Formatted at 2026-08-07 08:51:34 on dist-test-slave-6zbq"
server_key: "4c082adc4e843433dd9b517b088afc2e"
server_key_iv: "254fb064925219aefed2062d1d8bc5a1"
server_key_version: "encryptionkey@0"
I20260807 08:51:34.323668 3072 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786092453486433-4734-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786092453486433-4734-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786092453486433-4734-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:51:34.338204 3072 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:51:34.340104 3072 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:51:34.340231 3072 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:51:34.343987 3072 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43189
I20260807 08:51:34.344018 3167 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43189 every 8 connection(s)
I20260807 08:51:34.344309 3072 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
I20260807 08:51:34.345201 3168 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:51:34.347302 3168 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a2414f2b5704b64849238d709a16b39: Bootstrap starting.
I20260807 08:51:34.347786 3168 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8a2414f2b5704b64849238d709a16b39: Neither blocks nor log segments found. Creating new log.
I20260807 08:51:34.348021 3168 log.cc:824] T 00000000000000000000000000000000 P 8a2414f2b5704b64849238d709a16b39: Log is configured to *not* fsync() on all Append() calls
I20260807 08:51:34.348620 3168 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a2414f2b5704b64849238d709a16b39: No bootstrap required, opened a new log
I20260807 08:51:34.348676 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 3072
I20260807 08:51:34.348795 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786092453486433-4734-0/minicluster-data/master-0/wal/instance
I20260807 08:51:34.348964 4734 external_mini_cluster.cc:1428] Setting key 662200f664ae1e19f7b17b5122a0d604
I20260807 08:51:34.350482 3168 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8a2414f2b5704b64849238d709a16b39 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a2414f2b5704b64849238d709a16b39" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43189 } }
I20260807 08:51:34.350615 3168 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8a2414f2b5704b64849238d709a16b39 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:51:34.350651 3168 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8a2414f2b5704b64849238d709a16b39 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a2414f2b5704b64849238d709a16b39, State: Initialized, Role: FOLLOWER
I20260807 08:51:34.350822 3168 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8a2414f2b5704b64849238d709a16b39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a2414f2b5704b64849238d709a16b39" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43189 } }
I20260807 08:51:34.350910 3168 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8a2414f2b5704b64849238d709a16b39 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:51:34.350950 3168 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8a2414f2b5704b64849238d709a16b39 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:51:34.350997 3168 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8a2414f2b5704b64849238d709a16b39 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:51:34.351483 3168 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8a2414f2b5704b64849238d709a16b39 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a2414f2b5704b64849238d709a16b39" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43189 } }
I20260807 08:51:34.351572 3168 leader_election.cc:302] T 00000000000000000000000000000000 P 8a2414f2b5704b64849238d709a16b39 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8a2414f2b5704b64849238d709a16b39; no voters:
I20260807 08:51:34.351725 3168 leader_election.cc:288] T 00000000000000000000000000000000 P 8a2414f2b5704b64849238d709a16b39 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:51:34.351756 3177 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8a2414f2b5704b64849238d709a16b39 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:51:34.351919 3177 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8a2414f2b5704b64849238d709a16b39 [term 1 LEADER]: Becoming Leader. State: Replica: 8a2414f2b5704b64849238d709a16b39, State: Running, Role: LEADER
I20260807 08:51:34.351953 3168 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8a2414f2b5704b64849238d709a16b39 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:51:34.352000 3177 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8a2414f2b5704b64849238d709a16b39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a2414f2b5704b64849238d709a16b39" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43189 } }
I20260807 08:51:34.352401 3178 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a2414f2b5704b64849238d709a16b39 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8a2414f2b5704b64849238d709a16b39" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a2414f2b5704b64849238d709a16b39" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43189 } } }
I20260807 08:51:34.352547 3180 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a2414f2b5704b64849238d709a16b39 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8a2414f2b5704b64849238d709a16b39. Latest consensus state: current_term: 1 leader_uuid: "8a2414f2b5704b64849238d709a16b39" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a2414f2b5704b64849238d709a16b39" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43189 } } }
I20260807 08:51:34.352638 3178 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a2414f2b5704b64849238d709a16b39 [sys.catalog]: This master's current role is: LEADER
I20260807 08:51:34.352723 3180 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a2414f2b5704b64849238d709a16b39 [sys.catalog]: This master's current role is: LEADER
I20260807 08:51:34.352947 3190 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:51:34.353432 3190 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:51:34.354966 3190 catalog_manager.cc:1347] Generated new cluster ID: 6bf637fc73c443b294061e4f56f64c87
I20260807 08:51:34.355012 3190 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:51:34.355343 3176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:34.349393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47341 (local address 127.0.0.1:43189)
0807 08:51:34.349555 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:34.349562 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:34.349586 (+ 24us) server_negotiation.cc:407] Connection header received
0807 08:51:34.350457 (+ 871us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:34.350463 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:34.350517 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:34.350624 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:51:34.350886 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:34.352230 (+ 1344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:34.353270 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:34.353834 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:34.353933 (+ 99us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:34.354700 (+ 767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:51:34.354703 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:51:34.354704 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:51:34.354889 (+ 185us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:51:34.354925 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:51:34.354940 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:51:34.355021 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:34.355110 (+ 89us) server_negotiation.cc:299] Negotiation successful
0807 08:51:34.355137 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":74,"threads_started":1}
I20260807 08:51:34.356194 4734 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 08:51:34.364208 3190 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:51:34.364655 3190 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:51:34.367383 3190 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8a2414f2b5704b64849238d709a16b39: Generated new TSK 0
I20260807 08:51:34.367532 3190 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:51:34.367852 3176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:34.357514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38551 (local address 127.0.0.1:43189)
0807 08:51:34.357542 (+ 28us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:34.357544 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:34.364009 (+ 6465us) server_negotiation.cc:407] Connection header received
0807 08:51:34.364065 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:34.364066 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:34.364076 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:34.364104 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:51:34.364870 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:34.365561 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:34.366567 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:34.366871 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:34.366927 (+ 56us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:34.367568 (+ 641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:51:34.367569 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:51:34.367570 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:51:34.367575 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:51:34.367588 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:51:34.367591 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:51:34.367652 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:34.367772 (+ 120us) server_negotiation.cc:299] Negotiation successful
0807 08:51:34.367782 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260807 08:51:34.369439 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 3072
W20260807 08:51:34.381582 3055 connection.cc:535] client connection to 127.0.0.1:43189 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43189): Connection reset by peer (error 104)
2026-08-07T08:51:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1382 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786092453486433-4734-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:51:34 dist-test-slave-6zbq krb5kdc[3227](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:51:34 dist-test-slave-6zbq krb5kdc[3227](info): set up 2 sockets
Aug 07 08:51:34 dist-test-slave-6zbq krb5kdc[3227](info): commencing operation
krb5kdc: starting...
W20260807 08:51:35.488960 4734 mini_kdc.cc:121] Time spent starting KDC: real 1.094s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786092453486433-4734-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786092453486433-4734-0/krb5kdc/test-user.keytab.
Aug 07 08:51:35 dist-test-slave-6zbq krb5kdc[3227](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092695, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:51:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:51:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786092453486433-4734-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786092453486433-4734-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 08:51:35.594468 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786092453486433-4734-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786092453486433-4734-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786092453486433-4734-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43995
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786092453486433-4734-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43995
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:51:35.671619 3242 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:51:35.671797 3242 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:51:35.671818 3242 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:51:35.672851 3242 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:51:35.672883 3242 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:51:35.672894 3242 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:51:35.672905 3242 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:51:35.672919 3242 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:51:35.674250 3242 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786092453486433-4734-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786092453486433-4734-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43995
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786092453486433-4734-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43995
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3242
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786092453486433-4734-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:51:35.674458 3242 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:51:35.674623 3242 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 08:51:35.678967 3242 server_base.cc:1034] running on GCE node
W20260807 08:51:35.678932 3251 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:51:35.678960 3253 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:51:35.678937 3250 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:51:35.679376 3242 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:51:35.679620 3242 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:51:35.680748 3242 hybrid_clock.cc:648] HybridClock initialized: now 1786092695680736 us; error 28 us; skew 500 ppm
Aug 07 08:51:35 dist-test-slave-6zbq krb5kdc[3227](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092695, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:51:35.682924 3242 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 08:51:35.683310 3242 webserver.cc:460] Webserver started at http://127.0.0.1:36359/ using document root <none> and password file <none>
I20260807 08:51:35.683488 3242 fs_manager.cc:359] Metadata directory not provided
I20260807 08:51:35.683532 3242 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:51:35.683648 3242 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:51:35.684553 3242 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786092453486433-4734-0/minicluster-data/master-0/data/instance:
uuid: "0454889cc15c454895e59f9730e02abe"
format_stamp: "Formatted at 2026-08-07 08:51:35 on dist-test-slave-6zbq"
server_key: "7dc7c8bd0daf846705a12efbdcb016fb"
server_key_iv: "82a6081c89ac3e7bc5ad2632ea304601"
server_key_version: "encryptionkey@0"
I20260807 08:51:35.684863 3242 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786092453486433-4734-0/minicluster-data/master-0/wal/instance:
uuid: "0454889cc15c454895e59f9730e02abe"
format_stamp: "Formatted at 2026-08-07 08:51:35 on dist-test-slave-6zbq"
server_key: "7dc7c8bd0daf846705a12efbdcb016fb"
server_key_iv: "82a6081c89ac3e7bc5ad2632ea304601"
server_key_version: "encryptionkey@0"
I20260807 08:51:35.686002 3242 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 08:51:35.686646 3265 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:51:35.686790 3242 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 08:51:35.686868 3242 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786092453486433-4734-0/minicluster-data/master-0/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786092453486433-4734-0/minicluster-data/master-0/wal
uuid: "0454889cc15c454895e59f9730e02abe"
format_stamp: "Formatted at 2026-08-07 08:51:35 on dist-test-slave-6zbq"
server_key: "7dc7c8bd0daf846705a12efbdcb016fb"
server_key_iv: "82a6081c89ac3e7bc5ad2632ea304601"
server_key_version: "encryptionkey@0"
I20260807 08:51:35.686924 3242 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786092453486433-4734-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786092453486433-4734-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786092453486433-4734-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:51:35.689913 3242 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:51:35.691466 3242 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:51:35.691586 3242 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:51:35.695276 3242 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43995
I20260807 08:51:35.695309 3349 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43995 every 8 connection(s)
I20260807 08:51:35.695565 3242 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
I20260807 08:51:35.696393 3350 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:51:35.697932 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 3242
I20260807 08:51:35.698022 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786092453486433-4734-0/minicluster-data/master-0/wal/instance
I20260807 08:51:35.698166 4734 external_mini_cluster.cc:1428] Setting key 57ede2972785ae4d2f8b04d1f69a3cd1
I20260807 08:51:35.698506 3350 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0454889cc15c454895e59f9730e02abe: Bootstrap starting.
I20260807 08:51:35.699146 3350 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0454889cc15c454895e59f9730e02abe: Neither blocks nor log segments found. Creating new log.
I20260807 08:51:35.699471 3350 log.cc:824] T 00000000000000000000000000000000 P 0454889cc15c454895e59f9730e02abe: Log is configured to *not* fsync() on all Append() calls
I20260807 08:51:35.700177 3350 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0454889cc15c454895e59f9730e02abe: No bootstrap required, opened a new log
I20260807 08:51:35.700459 3353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:35.698606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55557 (local address 127.0.0.1:43995)
0807 08:51:35.698807 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:35.698816 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:35.698853 (+ 37us) server_negotiation.cc:407] Connection header received
0807 08:51:35.699365 (+ 512us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:35.699373 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:35.699416 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:35.699559 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:51:35.700076 (+ 517us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:51:35.700079 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:51:35.700079 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:51:35.700084 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:51:35.700124 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:51:35.700137 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:51:35.700218 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:35.700258 (+ 40us) server_negotiation.cc:299] Negotiation successful
0807 08:51:35.700299 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":89,"threads_started":1}
I20260807 08:51:35.701345 4734 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 08:51:35.702136 3350 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0454889cc15c454895e59f9730e02abe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0454889cc15c454895e59f9730e02abe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43995 } }
I20260807 08:51:35.702291 3350 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0454889cc15c454895e59f9730e02abe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:51:35.702332 3350 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0454889cc15c454895e59f9730e02abe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0454889cc15c454895e59f9730e02abe, State: Initialized, Role: FOLLOWER
I20260807 08:51:35.702484 3350 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0454889cc15c454895e59f9730e02abe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0454889cc15c454895e59f9730e02abe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43995 } }
I20260807 08:51:35.702617 3350 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0454889cc15c454895e59f9730e02abe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:51:35.702665 3350 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0454889cc15c454895e59f9730e02abe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:51:35.702706 3350 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0454889cc15c454895e59f9730e02abe [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:51:35.703271 3350 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0454889cc15c454895e59f9730e02abe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0454889cc15c454895e59f9730e02abe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43995 } }
I20260807 08:51:35.703366 3350 leader_election.cc:302] T 00000000000000000000000000000000 P 0454889cc15c454895e59f9730e02abe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0454889cc15c454895e59f9730e02abe; no voters:
I20260807 08:51:35.703507 3353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:35.702547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35663 (local address 127.0.0.1:43995)
0807 08:51:35.702578 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:35.702580 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:35.702728 (+ 148us) server_negotiation.cc:407] Connection header received
0807 08:51:35.702787 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:35.702788 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:35.702792 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:35.702835 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:51:35.703297 (+ 462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:51:35.703298 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:51:35.703299 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:51:35.703302 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:51:35.703313 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:51:35.703316 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:51:35.703366 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:35.703439 (+ 73us) server_negotiation.cc:299] Negotiation successful
0807 08:51:35.703450 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260807 08:51:35.703514 3350 leader_election.cc:288] T 00000000000000000000000000000000 P 0454889cc15c454895e59f9730e02abe [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:51:35.703776 3350 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0454889cc15c454895e59f9730e02abe [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:51:35.703545 3363 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0454889cc15c454895e59f9730e02abe [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:51:35.704036 3363 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0454889cc15c454895e59f9730e02abe [term 1 LEADER]: Becoming Leader. State: Replica: 0454889cc15c454895e59f9730e02abe, State: Running, Role: LEADER
I20260807 08:51:35.704126 3363 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0454889cc15c454895e59f9730e02abe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0454889cc15c454895e59f9730e02abe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43995 } }
I20260807 08:51:35.704457 3365 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0454889cc15c454895e59f9730e02abe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0454889cc15c454895e59f9730e02abe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0454889cc15c454895e59f9730e02abe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43995 } } }
I20260807 08:51:35.704469 3367 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0454889cc15c454895e59f9730e02abe [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0454889cc15c454895e59f9730e02abe. Latest consensus state: current_term: 1 leader_uuid: "0454889cc15c454895e59f9730e02abe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0454889cc15c454895e59f9730e02abe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43995 } } }
I20260807 08:51:35.704618 3367 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0454889cc15c454895e59f9730e02abe [sys.catalog]: This master's current role is: LEADER
I20260807 08:51:35.704605 3365 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0454889cc15c454895e59f9730e02abe [sys.catalog]: This master's current role is: LEADER
W20260807 08:51:35.705421 3389 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0454889cc15c454895e59f9730e02abe: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 08:51:35.705464 3389 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 08:51:35.705500 3382 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:51:35.705667 3382 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:51:35.707002 3382 catalog_manager.cc:1347] Generated new cluster ID: 153ebbe91bd44a4aa531f23088141af1
I20260807 08:51:35.707041 3382 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:51:35.735656 3382 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:51:35.735791 3382 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:51:35.742751 3382 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0454889cc15c454895e59f9730e02abe: Generated new TSK 0
I20260807 08:51:35.742887 3382 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:51:35.751225 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 3242
2026-08-07T08:51:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1374 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786092453486433-4734-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:51:35 dist-test-slave-6zbq krb5kdc[3395](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:51:35 dist-test-slave-6zbq krb5kdc[3395](info): set up 2 sockets
Aug 07 08:51:35 dist-test-slave-6zbq krb5kdc[3395](info): commencing operation
krb5kdc: starting...
W20260807 08:51:36.860606 4734 mini_kdc.cc:121] Time spent starting KDC: real 1.091s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786092453486433-4734-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786092453486433-4734-0/krb5kdc/test-user.keytab.
Aug 07 08:51:36 dist-test-slave-6zbq krb5kdc[3395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092696, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:51:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:51:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786092453486433-4734-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786092453486433-4734-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 08:51:36.967042 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786092453486433-4734-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786092453486433-4734-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786092453486433-4734-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46319
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786092453486433-4734-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46319
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:51:37.045213 3414 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:51:37.045401 3414 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:51:37.045431 3414 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:51:37.046540 3414 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:51:37.046576 3414 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:51:37.046597 3414 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:51:37.046619 3414 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:51:37.046636 3414 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:51:37.048061 3414 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786092453486433-4734-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786092453486433-4734-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46319
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786092453486433-4734-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46319
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3414
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786092453486433-4734-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:51:37.048285 3414 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:51:37.048473 3414 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:51:37.052826 3424 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:51:37.052791 3426 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:51:37.052822 3414 server_base.cc:1034] running on GCE node
W20260807 08:51:37.052798 3423 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:51:37.053243 3414 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:51:37.053494 3414 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:51:37.054626 3414 hybrid_clock.cc:648] HybridClock initialized: now 1786092697054611 us; error 31 us; skew 500 ppm
Aug 07 08:51:37 dist-test-slave-6zbq krb5kdc[3395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092697, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:51:37.056865 3414 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 08:51:37.057237 3414 webserver.cc:460] Webserver started at http://127.0.0.1:44461/ using document root <none> and password file <none>
I20260807 08:51:37.057397 3414 fs_manager.cc:359] Metadata directory not provided
I20260807 08:51:37.057435 3414 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:51:37.057518 3414 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:51:37.058468 3414 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786092453486433-4734-0/minicluster-data/master-0/data/instance:
uuid: "fd4e6bb43b2d462abd02b4deb1b297f9"
format_stamp: "Formatted at 2026-08-07 08:51:37 on dist-test-slave-6zbq"
server_key: "6665886be92e606dc82f622da5d8794e"
server_key_iv: "17f81b3bdbf5ca77107f3eb74c60ebe4"
server_key_version: "encryptionkey@0"
I20260807 08:51:37.058755 3414 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786092453486433-4734-0/minicluster-data/master-0/wal/instance:
uuid: "fd4e6bb43b2d462abd02b4deb1b297f9"
format_stamp: "Formatted at 2026-08-07 08:51:37 on dist-test-slave-6zbq"
server_key: "6665886be92e606dc82f622da5d8794e"
server_key_iv: "17f81b3bdbf5ca77107f3eb74c60ebe4"
server_key_version: "encryptionkey@0"
I20260807 08:51:37.059877 3414 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 08:51:37.060518 3436 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:51:37.060683 3414 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 08:51:37.060763 3414 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786092453486433-4734-0/minicluster-data/master-0/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786092453486433-4734-0/minicluster-data/master-0/wal
uuid: "fd4e6bb43b2d462abd02b4deb1b297f9"
format_stamp: "Formatted at 2026-08-07 08:51:37 on dist-test-slave-6zbq"
server_key: "6665886be92e606dc82f622da5d8794e"
server_key_iv: "17f81b3bdbf5ca77107f3eb74c60ebe4"
server_key_version: "encryptionkey@0"
I20260807 08:51:37.060817 3414 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786092453486433-4734-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786092453486433-4734-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786092453486433-4734-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:51:37.067806 3414 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:51:37.069309 3414 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:51:37.069408 3414 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:51:37.073040 3414 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46319
I20260807 08:51:37.073082 3518 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46319 every 8 connection(s)
I20260807 08:51:37.073360 3414 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
I20260807 08:51:37.074225 3519 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:51:37.076284 3519 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd4e6bb43b2d462abd02b4deb1b297f9: Bootstrap starting.
I20260807 08:51:37.076793 3519 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fd4e6bb43b2d462abd02b4deb1b297f9: Neither blocks nor log segments found. Creating new log.
I20260807 08:51:37.077014 3519 log.cc:824] T 00000000000000000000000000000000 P fd4e6bb43b2d462abd02b4deb1b297f9: Log is configured to *not* fsync() on all Append() calls
I20260807 08:51:37.077586 3519 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd4e6bb43b2d462abd02b4deb1b297f9: No bootstrap required, opened a new log
I20260807 08:51:37.079144 3519 raft_consensus.cc:348] T 00000000000000000000000000000000 P fd4e6bb43b2d462abd02b4deb1b297f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd4e6bb43b2d462abd02b4deb1b297f9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46319 } }
I20260807 08:51:37.079268 3519 raft_consensus.cc:374] T 00000000000000000000000000000000 P fd4e6bb43b2d462abd02b4deb1b297f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:51:37.079295 3519 raft_consensus.cc:729] T 00000000000000000000000000000000 P fd4e6bb43b2d462abd02b4deb1b297f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd4e6bb43b2d462abd02b4deb1b297f9, State: Initialized, Role: FOLLOWER
I20260807 08:51:37.079393 3519 consensus_queue.cc:260] T 00000000000000000000000000000000 P fd4e6bb43b2d462abd02b4deb1b297f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd4e6bb43b2d462abd02b4deb1b297f9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46319 } }
I20260807 08:51:37.079466 3519 raft_consensus.cc:388] T 00000000000000000000000000000000 P fd4e6bb43b2d462abd02b4deb1b297f9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:51:37.079493 3519 raft_consensus.cc:482] T 00000000000000000000000000000000 P fd4e6bb43b2d462abd02b4deb1b297f9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:51:37.079523 3519 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fd4e6bb43b2d462abd02b4deb1b297f9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:51:37.079964 3519 raft_consensus.cc:504] T 00000000000000000000000000000000 P fd4e6bb43b2d462abd02b4deb1b297f9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd4e6bb43b2d462abd02b4deb1b297f9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46319 } }
I20260807 08:51:37.080045 3519 leader_election.cc:302] T 00000000000000000000000000000000 P fd4e6bb43b2d462abd02b4deb1b297f9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fd4e6bb43b2d462abd02b4deb1b297f9; no voters:
I20260807 08:51:37.080165 3519 leader_election.cc:288] T 00000000000000000000000000000000 P fd4e6bb43b2d462abd02b4deb1b297f9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:51:37.080202 3523 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fd4e6bb43b2d462abd02b4deb1b297f9 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:51:37.080331 3523 raft_consensus.cc:686] T 00000000000000000000000000000000 P fd4e6bb43b2d462abd02b4deb1b297f9 [term 1 LEADER]: Becoming Leader. State: Replica: fd4e6bb43b2d462abd02b4deb1b297f9, State: Running, Role: LEADER
I20260807 08:51:37.080382 3519 sys_catalog.cc:564] T 00000000000000000000000000000000 P fd4e6bb43b2d462abd02b4deb1b297f9 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:51:37.080426 3523 consensus_queue.cc:237] T 00000000000000000000000000000000 P fd4e6bb43b2d462abd02b4deb1b297f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd4e6bb43b2d462abd02b4deb1b297f9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46319 } }
I20260807 08:51:37.080881 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 3414
I20260807 08:51:37.080875 3525 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd4e6bb43b2d462abd02b4deb1b297f9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fd4e6bb43b2d462abd02b4deb1b297f9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd4e6bb43b2d462abd02b4deb1b297f9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46319 } } }
I20260807 08:51:37.080946 3525 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd4e6bb43b2d462abd02b4deb1b297f9 [sys.catalog]: This master's current role is: LEADER
I20260807 08:51:37.080987 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786092453486433-4734-0/minicluster-data/master-0/wal/instance
I20260807 08:51:37.081171 4734 external_mini_cluster.cc:1428] Setting key 4c4fa241c3044a47e20548078ff25364
I20260807 08:51:37.081454 3528 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd4e6bb43b2d462abd02b4deb1b297f9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fd4e6bb43b2d462abd02b4deb1b297f9. Latest consensus state: current_term: 1 leader_uuid: "fd4e6bb43b2d462abd02b4deb1b297f9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd4e6bb43b2d462abd02b4deb1b297f9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46319 } } }
I20260807 08:51:37.081573 3528 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd4e6bb43b2d462abd02b4deb1b297f9 [sys.catalog]: This master's current role is: LEADER
I20260807 08:51:37.081598 3540 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:51:37.082108 3540 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:51:37.083592 3540 catalog_manager.cc:1347] Generated new cluster ID: 472ea65c5be048968859b69de73767a4
I20260807 08:51:37.083643 3540 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 08:51:37 dist-test-slave-6zbq krb5kdc[3395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092696, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 08:51:37.090268 3562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:37.081792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36187 (local address 127.0.0.1:46319)
0807 08:51:37.081941 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:37.081950 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:37.082042 (+ 92us) server_negotiation.cc:407] Connection header received
0807 08:51:37.082190 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:37.082197 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:37.082255 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:37.082397 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:51:37.083069 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:37.084119 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:37.084838 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:37.085250 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:37.085341 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:37.087440 (+ 2099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:51:37.087444 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:51:37.087444 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:51:37.087456 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:51:37.088888 (+ 1432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:51:37.089192 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:51:37.089195 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:51:37.089195 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:51:37.089293 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:51:37.089463 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:51:37.089464 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:51:37.089464 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:51:37.089686 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:51:37.089834 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:37.089950 (+ 116us) server_negotiation.cc:299] Negotiation successful
0807 08:51:37.090081 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":103,"thread_start_us":77,"threads_started":1}
I20260807 08:51:37.091173 4734 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 08:51:37.091650 3540 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:51:37.092293 3540 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:51:37.098112 3540 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fd4e6bb43b2d462abd02b4deb1b297f9: Generated new TSK 0
I20260807 08:51:37.098286 3540 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:51:37.103219 3562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:37.094550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45725 (local address 127.0.0.1:46319)
0807 08:51:37.094584 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:37.094586 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:37.097925 (+ 3339us) server_negotiation.cc:407] Connection header received
0807 08:51:37.097988 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:37.097989 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:37.098000 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:37.098054 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:51:37.098654 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:37.099126 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:37.099803 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:37.100014 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:37.100065 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:37.101114 (+ 1049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:51:37.101115 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:51:37.101116 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:51:37.101122 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:51:37.102254 (+ 1132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:51:37.102572 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:51:37.102573 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:51:37.102573 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:51:37.102625 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:51:37.102787 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:51:37.102787 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:51:37.102787 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:51:37.102884 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:51:37.102965 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:37.103065 (+ 100us) server_negotiation.cc:299] Negotiation successful
0807 08:51:37.103137 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 08:51:37.105192 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 3414
2026-08-07T08:51:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1364 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786092453486433-4734-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:51:37 dist-test-slave-6zbq krb5kdc[3576](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:51:37 dist-test-slave-6zbq krb5kdc[3576](info): set up 2 sockets
Aug 07 08:51:37 dist-test-slave-6zbq krb5kdc[3576](info): commencing operation
krb5kdc: starting...
W20260807 08:51:38.225925 4734 mini_kdc.cc:121] Time spent starting KDC: real 1.092s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786092453486433-4734-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786092453486433-4734-0/krb5kdc/test-user.keytab.
Aug 07 08:51:38 dist-test-slave-6zbq krb5kdc[3576](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092698, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:51:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:51:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786092453486433-4734-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786092453486433-4734-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 08:51:38.330801 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786092453486433-4734-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786092453486433-4734-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786092453486433-4734-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36177
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786092453486433-4734-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36177
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:51:38.411676 3599 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:51:38.411872 3599 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:51:38.411893 3599 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:51:38.412988 3599 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:51:38.413028 3599 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:51:38.413043 3599 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:51:38.413053 3599 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:51:38.413076 3599 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:51:38.414530 3599 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786092453486433-4734-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786092453486433-4734-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36177
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786092453486433-4734-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36177
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3599
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786092453486433-4734-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:51:38.414758 3599 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:51:38.414954 3599 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:51:38.419369 3608 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:51:38.419379 3605 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:51:38.419405 3599 server_base.cc:1034] running on GCE node
W20260807 08:51:38.419421 3606 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:51:38.419765 3599 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:51:38.420032 3599 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:51:38.421196 3599 hybrid_clock.cc:648] HybridClock initialized: now 1786092698421181 us; error 30 us; skew 500 ppm
Aug 07 08:51:38 dist-test-slave-6zbq krb5kdc[3576](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092698, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:51:38.423442 3599 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 08:51:38.423888 3599 webserver.cc:460] Webserver started at http://127.0.0.1:35151/ using document root <none> and password file <none>
I20260807 08:51:38.424084 3599 fs_manager.cc:359] Metadata directory not provided
I20260807 08:51:38.424125 3599 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:51:38.424252 3599 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:51:38.425192 3599 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786092453486433-4734-0/minicluster-data/master-0/data/instance:
uuid: "4a8c85b2f6344b62b78fe08731ef912a"
format_stamp: "Formatted at 2026-08-07 08:51:38 on dist-test-slave-6zbq"
server_key: "6001b32767948bb1ce0114e6f32bc4b0"
server_key_iv: "fc302356afd7c5002d810b5f5fedeb21"
server_key_version: "encryptionkey@0"
I20260807 08:51:38.425516 3599 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786092453486433-4734-0/minicluster-data/master-0/wal/instance:
uuid: "4a8c85b2f6344b62b78fe08731ef912a"
format_stamp: "Formatted at 2026-08-07 08:51:38 on dist-test-slave-6zbq"
server_key: "6001b32767948bb1ce0114e6f32bc4b0"
server_key_iv: "fc302356afd7c5002d810b5f5fedeb21"
server_key_version: "encryptionkey@0"
I20260807 08:51:38.426720 3599 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 08:51:38.427399 3617 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:51:38.427541 3599 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 08:51:38.427623 3599 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786092453486433-4734-0/minicluster-data/master-0/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786092453486433-4734-0/minicluster-data/master-0/wal
uuid: "4a8c85b2f6344b62b78fe08731ef912a"
format_stamp: "Formatted at 2026-08-07 08:51:38 on dist-test-slave-6zbq"
server_key: "6001b32767948bb1ce0114e6f32bc4b0"
server_key_iv: "fc302356afd7c5002d810b5f5fedeb21"
server_key_version: "encryptionkey@0"
I20260807 08:51:38.427688 3599 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786092453486433-4734-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786092453486433-4734-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786092453486433-4734-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:51:38.437273 3599 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:51:38.438812 3599 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:51:38.438923 3599 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:51:38.442680 3599 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36177
I20260807 08:51:38.442709 3701 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36177 every 8 connection(s)
I20260807 08:51:38.442994 3599 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
I20260807 08:51:38.443830 3702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:51:38.444759 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 3599
I20260807 08:51:38.444880 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786092453486433-4734-0/minicluster-data/master-0/wal/instance
I20260807 08:51:38.445050 4734 external_mini_cluster.cc:1428] Setting key 4a2b990d4dbea19be42b3eccd901ee9a
I20260807 08:51:38.446230 3702 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a8c85b2f6344b62b78fe08731ef912a: Bootstrap starting.
I20260807 08:51:38.446848 3702 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4a8c85b2f6344b62b78fe08731ef912a: Neither blocks nor log segments found. Creating new log.
I20260807 08:51:38.447108 3702 log.cc:824] T 00000000000000000000000000000000 P 4a8c85b2f6344b62b78fe08731ef912a: Log is configured to *not* fsync() on all Append() calls
I20260807 08:51:38.447772 3702 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a8c85b2f6344b62b78fe08731ef912a: No bootstrap required, opened a new log
I20260807 08:51:38.449856 3702 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4a8c85b2f6344b62b78fe08731ef912a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a8c85b2f6344b62b78fe08731ef912a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36177 } }
I20260807 08:51:38.450016 3702 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4a8c85b2f6344b62b78fe08731ef912a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:51:38.450070 3702 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4a8c85b2f6344b62b78fe08731ef912a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a8c85b2f6344b62b78fe08731ef912a, State: Initialized, Role: FOLLOWER
I20260807 08:51:38.450197 3702 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4a8c85b2f6344b62b78fe08731ef912a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a8c85b2f6344b62b78fe08731ef912a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36177 } }
I20260807 08:51:38.450255 3708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:38.445504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57907 (local address 127.0.0.1:36177)
0807 08:51:38.445654 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:38.445662 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:38.445686 (+ 24us) server_negotiation.cc:407] Connection header received
0807 08:51:38.446099 (+ 413us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:38.446106 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:38.446148 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:38.446256 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:51:38.446513 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:38.447520 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:38.448536 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:38.448924 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:38.448997 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:38.449598 (+ 601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:51:38.449602 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:51:38.449602 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:51:38.449751 (+ 149us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:51:38.449779 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:51:38.449789 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:51:38.449832 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:38.449922 (+ 90us) server_negotiation.cc:299] Negotiation successful
0807 08:51:38.449968 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":61,"threads_started":1}
I20260807 08:51:38.450258 3702 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4a8c85b2f6344b62b78fe08731ef912a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:51:38.450362 3702 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4a8c85b2f6344b62b78fe08731ef912a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:51:38.450407 3702 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4a8c85b2f6344b62b78fe08731ef912a [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:51:38.450995 3702 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4a8c85b2f6344b62b78fe08731ef912a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a8c85b2f6344b62b78fe08731ef912a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36177 } }
I20260807 08:51:38.451093 3702 leader_election.cc:302] T 00000000000000000000000000000000 P 4a8c85b2f6344b62b78fe08731ef912a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4a8c85b2f6344b62b78fe08731ef912a; no voters:
I20260807 08:51:38.451113 4734 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 08:51:38.451236 3702 leader_election.cc:288] T 00000000000000000000000000000000 P 4a8c85b2f6344b62b78fe08731ef912a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:51:38.451264 3710 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4a8c85b2f6344b62b78fe08731ef912a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:51:38.451368 3710 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4a8c85b2f6344b62b78fe08731ef912a [term 1 LEADER]: Becoming Leader. State: Replica: 4a8c85b2f6344b62b78fe08731ef912a, State: Running, Role: LEADER
I20260807 08:51:38.451468 3702 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4a8c85b2f6344b62b78fe08731ef912a [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:51:38.451493 3710 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4a8c85b2f6344b62b78fe08731ef912a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a8c85b2f6344b62b78fe08731ef912a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36177 } }
I20260807 08:51:38.451807 3710 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a8c85b2f6344b62b78fe08731ef912a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4a8c85b2f6344b62b78fe08731ef912a. Latest consensus state: current_term: 1 leader_uuid: "4a8c85b2f6344b62b78fe08731ef912a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a8c85b2f6344b62b78fe08731ef912a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36177 } } }
I20260807 08:51:38.451930 3710 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a8c85b2f6344b62b78fe08731ef912a [sys.catalog]: This master's current role is: LEADER
I20260807 08:51:38.452119 3711 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a8c85b2f6344b62b78fe08731ef912a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4a8c85b2f6344b62b78fe08731ef912a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a8c85b2f6344b62b78fe08731ef912a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36177 } } }
I20260807 08:51:38.452229 3711 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a8c85b2f6344b62b78fe08731ef912a [sys.catalog]: This master's current role is: LEADER
I20260807 08:51:38.452385 3724 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:51:38.452932 3724 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:51:38.454469 3724 catalog_manager.cc:1347] Generated new cluster ID: 026c0830643f4728b3faf318b7b332d7
I20260807 08:51:38.454520 3724 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:51:38.456017 3708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:38.452894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33953 (local address 127.0.0.1:36177)
0807 08:51:38.452926 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:38.452927 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:38.453164 (+ 237us) server_negotiation.cc:407] Connection header received
0807 08:51:38.453259 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:38.453260 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:38.453269 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:38.453313 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:51:38.453420 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:38.454041 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:38.454874 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:38.455155 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:38.455187 (+ 32us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:38.455808 (+ 621us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:51:38.455809 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:51:38.455809 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:51:38.455813 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:51:38.455825 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:51:38.455829 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:51:38.455851 (+ 22us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:38.455931 (+ 80us) server_negotiation.cc:299] Negotiation successful
0807 08:51:38.455949 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260807 08:51:38.467481 3724 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:51:38.467947 3724 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:51:38.474277 3724 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4a8c85b2f6344b62b78fe08731ef912a: Generated new TSK 0
I20260807 08:51:38.474447 3724 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:51:38.479650 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 3599
2026-08-07T08:51:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1372 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786092453486433-4734-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:51:38 dist-test-slave-6zbq krb5kdc[3755](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:51:38 dist-test-slave-6zbq krb5kdc[3755](info): set up 2 sockets
Aug 07 08:51:38 dist-test-slave-6zbq krb5kdc[3755](info): commencing operation
krb5kdc: starting...
W20260807 08:51:39.612450 4734 mini_kdc.cc:121] Time spent starting KDC: real 1.106s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786092453486433-4734-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786092453486433-4734-0/krb5kdc/test-user.keytab.
Aug 07 08:51:39 dist-test-slave-6zbq krb5kdc[3755](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092699, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:51:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:51:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786092453486433-4734-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786092453486433-4734-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 08:51:39.720108 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786092453486433-4734-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786092453486433-4734-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786092453486433-4734-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40455
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786092453486433-4734-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40455
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:51:39.801095 3772 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:51:39.801321 3772 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:51:39.801347 3772 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:51:39.802557 3772 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:51:39.802599 3772 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:51:39.802613 3772 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:51:39.802624 3772 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:51:39.802639 3772 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:51:39.804159 3772 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786092453486433-4734-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786092453486433-4734-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40455
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786092453486433-4734-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40455
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3772
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786092453486433-4734-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:51:39.804390 3772 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:51:39.804598 3772 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:51:39.809125 3790 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:51:39.809222 3772 server_base.cc:1034] running on GCE node
W20260807 08:51:39.809135 3784 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:51:39.809131 3786 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:51:39.809583 3772 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:51:39.809861 3772 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:51:39.810995 3772 hybrid_clock.cc:648] HybridClock initialized: now 1786092699810980 us; error 31 us; skew 500 ppm
Aug 07 08:51:39 dist-test-slave-6zbq krb5kdc[3755](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092699, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:51:39.819103 3772 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 08:51:39.819506 3772 webserver.cc:460] Webserver started at http://127.0.0.1:40487/ using document root <none> and password file <none>
I20260807 08:51:39.819679 3772 fs_manager.cc:359] Metadata directory not provided
I20260807 08:51:39.819720 3772 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:51:39.819813 3772 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:51:39.820796 3772 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786092453486433-4734-0/minicluster-data/master-0/data/instance:
uuid: "e5a315ab9f4149c9b8b6727dac5fa126"
format_stamp: "Formatted at 2026-08-07 08:51:39 on dist-test-slave-6zbq"
server_key: "529fca6cd0b4226a691aa3f4a1f4bd7d"
server_key_iv: "ca7252918f9ea08b05a491fa03b83c47"
server_key_version: "encryptionkey@0"
I20260807 08:51:39.821102 3772 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786092453486433-4734-0/minicluster-data/master-0/wal/instance:
uuid: "e5a315ab9f4149c9b8b6727dac5fa126"
format_stamp: "Formatted at 2026-08-07 08:51:39 on dist-test-slave-6zbq"
server_key: "529fca6cd0b4226a691aa3f4a1f4bd7d"
server_key_iv: "ca7252918f9ea08b05a491fa03b83c47"
server_key_version: "encryptionkey@0"
I20260807 08:51:39.822248 3772 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260807 08:51:39.822934 3800 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:51:39.823081 3772 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 08:51:39.823163 3772 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786092453486433-4734-0/minicluster-data/master-0/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786092453486433-4734-0/minicluster-data/master-0/wal
uuid: "e5a315ab9f4149c9b8b6727dac5fa126"
format_stamp: "Formatted at 2026-08-07 08:51:39 on dist-test-slave-6zbq"
server_key: "529fca6cd0b4226a691aa3f4a1f4bd7d"
server_key_iv: "ca7252918f9ea08b05a491fa03b83c47"
server_key_version: "encryptionkey@0"
I20260807 08:51:39.823231 3772 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786092453486433-4734-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786092453486433-4734-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786092453486433-4734-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:51:39.826431 3772 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:51:39.828303 3772 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:51:39.828428 3772 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:51:39.832082 3772 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40455
I20260807 08:51:39.832129 3883 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40455 every 8 connection(s)
I20260807 08:51:39.832402 3772 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
I20260807 08:51:39.833303 3885 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:51:39.834429 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 3772
I20260807 08:51:39.834534 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786092453486433-4734-0/minicluster-data/master-0/wal/instance
I20260807 08:51:39.834687 4734 external_mini_cluster.cc:1428] Setting key 78b5e046fa9e0840433089de8bde9757
I20260807 08:51:39.835819 3885 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5a315ab9f4149c9b8b6727dac5fa126: Bootstrap starting.
I20260807 08:51:39.836505 3885 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e5a315ab9f4149c9b8b6727dac5fa126: Neither blocks nor log segments found. Creating new log.
I20260807 08:51:39.836817 3885 log.cc:824] T 00000000000000000000000000000000 P e5a315ab9f4149c9b8b6727dac5fa126: Log is configured to *not* fsync() on all Append() calls
I20260807 08:51:39.836958 3893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:39.835120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50953 (local address 127.0.0.1:40455)
0807 08:51:39.835305 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:39.835314 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:39.835349 (+ 35us) server_negotiation.cc:407] Connection header received
0807 08:51:39.835871 (+ 522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:39.835880 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:39.835913 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:39.836055 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:51:39.836555 (+ 500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:51:39.836558 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:51:39.836559 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:51:39.836563 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:51:39.836605 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:51:39.836621 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:51:39.836722 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:39.836753 (+ 31us) server_negotiation.cc:299] Negotiation successful
0807 08:51:39.836791 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":85,"threads_started":1}
I20260807 08:51:39.837525 3885 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5a315ab9f4149c9b8b6727dac5fa126: No bootstrap required, opened a new log
I20260807 08:51:39.837775 4734 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 08:51:39.839522 3885 raft_consensus.cc:348] T 00000000000000000000000000000000 P e5a315ab9f4149c9b8b6727dac5fa126 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5a315ab9f4149c9b8b6727dac5fa126" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40455 } }
I20260807 08:51:39.839646 3885 raft_consensus.cc:374] T 00000000000000000000000000000000 P e5a315ab9f4149c9b8b6727dac5fa126 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:51:39.839677 3885 raft_consensus.cc:729] T 00000000000000000000000000000000 P e5a315ab9f4149c9b8b6727dac5fa126 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5a315ab9f4149c9b8b6727dac5fa126, State: Initialized, Role: FOLLOWER
I20260807 08:51:39.839790 3885 consensus_queue.cc:260] T 00000000000000000000000000000000 P e5a315ab9f4149c9b8b6727dac5fa126 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5a315ab9f4149c9b8b6727dac5fa126" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40455 } }
I20260807 08:51:39.839862 3885 raft_consensus.cc:388] T 00000000000000000000000000000000 P e5a315ab9f4149c9b8b6727dac5fa126 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:51:39.839891 3885 raft_consensus.cc:482] T 00000000000000000000000000000000 P e5a315ab9f4149c9b8b6727dac5fa126 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:51:39.839965 3893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:39.839063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55595 (local address 127.0.0.1:40455)
0807 08:51:39.839111 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:39.839113 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:39.839152 (+ 39us) server_negotiation.cc:407] Connection header received
0807 08:51:39.839207 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:39.839207 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:39.839212 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:39.839287 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:51:39.839739 (+ 452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:51:39.839740 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:51:39.839741 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:51:39.839745 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:51:39.839757 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:51:39.839761 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:51:39.839801 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:39.839877 (+ 76us) server_negotiation.cc:299] Negotiation successful
0807 08:51:39.839888 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260807 08:51:39.839977 3885 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e5a315ab9f4149c9b8b6727dac5fa126 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:51:39.840510 3885 raft_consensus.cc:504] T 00000000000000000000000000000000 P e5a315ab9f4149c9b8b6727dac5fa126 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5a315ab9f4149c9b8b6727dac5fa126" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40455 } }
I20260807 08:51:39.840605 3885 leader_election.cc:302] T 00000000000000000000000000000000 P e5a315ab9f4149c9b8b6727dac5fa126 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e5a315ab9f4149c9b8b6727dac5fa126; no voters:
I20260807 08:51:39.840767 3885 leader_election.cc:288] T 00000000000000000000000000000000 P e5a315ab9f4149c9b8b6727dac5fa126 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:51:39.840795 3900 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e5a315ab9f4149c9b8b6727dac5fa126 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:51:39.840946 3900 raft_consensus.cc:686] T 00000000000000000000000000000000 P e5a315ab9f4149c9b8b6727dac5fa126 [term 1 LEADER]: Becoming Leader. State: Replica: e5a315ab9f4149c9b8b6727dac5fa126, State: Running, Role: LEADER
I20260807 08:51:39.840983 3885 sys_catalog.cc:564] T 00000000000000000000000000000000 P e5a315ab9f4149c9b8b6727dac5fa126 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:51:39.841058 3900 consensus_queue.cc:237] T 00000000000000000000000000000000 P e5a315ab9f4149c9b8b6727dac5fa126 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5a315ab9f4149c9b8b6727dac5fa126" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40455 } }
I20260807 08:51:39.841439 3901 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5a315ab9f4149c9b8b6727dac5fa126 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e5a315ab9f4149c9b8b6727dac5fa126" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5a315ab9f4149c9b8b6727dac5fa126" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40455 } } }
I20260807 08:51:39.841449 3902 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5a315ab9f4149c9b8b6727dac5fa126 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e5a315ab9f4149c9b8b6727dac5fa126. Latest consensus state: current_term: 1 leader_uuid: "e5a315ab9f4149c9b8b6727dac5fa126" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5a315ab9f4149c9b8b6727dac5fa126" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40455 } } }
I20260807 08:51:39.841557 3901 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5a315ab9f4149c9b8b6727dac5fa126 [sys.catalog]: This master's current role is: LEADER
I20260807 08:51:39.841562 3902 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5a315ab9f4149c9b8b6727dac5fa126 [sys.catalog]: This master's current role is: LEADER
I20260807 08:51:39.841763 3913 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:51:39.842221 3913 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:51:39.843554 3913 catalog_manager.cc:1347] Generated new cluster ID: adca1c17542e484796bf9159e3882cbb
I20260807 08:51:39.843607 3913 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:51:39.885140 3913 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:51:39.885335 3913 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:51:39.896167 3913 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e5a315ab9f4149c9b8b6727dac5fa126: Generated new TSK 0
I20260807 08:51:39.896343 3913 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:51:39.905661 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 3772
2026-08-07T08:51:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1423 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786092453486433-4734-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:51:39 dist-test-slave-6zbq krb5kdc[3932](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:51:39 dist-test-slave-6zbq krb5kdc[3932](info): set up 2 sockets
Aug 07 08:51:39 dist-test-slave-6zbq krb5kdc[3932](info): commencing operation
krb5kdc: starting...
W20260807 08:51:41.029644 4734 mini_kdc.cc:121] Time spent starting KDC: real 1.100s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786092453486433-4734-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786092453486433-4734-0/krb5kdc/test-user.keytab.
Aug 07 08:51:41 dist-test-slave-6zbq krb5kdc[3932](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092701, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:51:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:51:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786092453486433-4734-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786092453486433-4734-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 08:51:41.138727 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786092453486433-4734-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786092453486433-4734-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786092453486433-4734-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43567
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786092453486433-4734-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43567
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:51:41.216976 3950 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:51:41.217150 3950 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:51:41.217170 3950 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:51:41.218214 3950 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:51:41.218245 3950 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:51:41.218257 3950 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:51:41.218268 3950 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:51:41.218282 3950 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:51:41.219646 3950 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786092453486433-4734-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786092453486433-4734-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43567
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786092453486433-4734-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43567
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3950
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786092453486433-4734-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:51:41.219830 3950 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:51:41.219985 3950 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:51:41.224325 3962 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:51:41.224314 3960 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:51:41.224390 3950 server_base.cc:1034] running on GCE node
W20260807 08:51:41.224325 3959 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:51:41.224726 3950 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:51:41.224958 3950 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:51:41.226090 3950 hybrid_clock.cc:648] HybridClock initialized: now 1786092701226075 us; error 26 us; skew 500 ppm
Aug 07 08:51:41 dist-test-slave-6zbq krb5kdc[3932](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092701, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:51:41.228296 3950 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 08:51:41.228657 3950 webserver.cc:460] Webserver started at http://127.0.0.1:35751/ using document root <none> and password file <none>
I20260807 08:51:41.228842 3950 fs_manager.cc:359] Metadata directory not provided
I20260807 08:51:41.228888 3950 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:51:41.229012 3950 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:51:41.229975 3950 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786092453486433-4734-0/minicluster-data/master-0/data/instance:
uuid: "0045c2909df44c0db614c7a1af15520c"
format_stamp: "Formatted at 2026-08-07 08:51:41 on dist-test-slave-6zbq"
server_key: "0e7e8820f145f375eb6adee4f46d59c4"
server_key_iv: "e5806860e5b02a5513947f56fe72d2d0"
server_key_version: "encryptionkey@0"
I20260807 08:51:41.230309 3950 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786092453486433-4734-0/minicluster-data/master-0/wal/instance:
uuid: "0045c2909df44c0db614c7a1af15520c"
format_stamp: "Formatted at 2026-08-07 08:51:41 on dist-test-slave-6zbq"
server_key: "0e7e8820f145f375eb6adee4f46d59c4"
server_key_iv: "e5806860e5b02a5513947f56fe72d2d0"
server_key_version: "encryptionkey@0"
I20260807 08:51:41.231501 3950 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 08:51:41.232102 3974 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:51:41.232280 3950 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 08:51:41.232362 3950 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786092453486433-4734-0/minicluster-data/master-0/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786092453486433-4734-0/minicluster-data/master-0/wal
uuid: "0045c2909df44c0db614c7a1af15520c"
format_stamp: "Formatted at 2026-08-07 08:51:41 on dist-test-slave-6zbq"
server_key: "0e7e8820f145f375eb6adee4f46d59c4"
server_key_iv: "e5806860e5b02a5513947f56fe72d2d0"
server_key_version: "encryptionkey@0"
I20260807 08:51:41.232419 3950 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786092453486433-4734-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786092453486433-4734-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786092453486433-4734-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:51:41.249830 3950 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:51:41.251374 3950 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:51:41.251492 3950 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:51:41.255136 3950 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43567
I20260807 08:51:41.255182 4050 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43567 every 8 connection(s)
I20260807 08:51:41.255465 3950 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
I20260807 08:51:41.256337 4053 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:51:41.258391 4053 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0045c2909df44c0db614c7a1af15520c: Bootstrap starting.
I20260807 08:51:41.258869 4053 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0045c2909df44c0db614c7a1af15520c: Neither blocks nor log segments found. Creating new log.
I20260807 08:51:41.259109 4053 log.cc:824] T 00000000000000000000000000000000 P 0045c2909df44c0db614c7a1af15520c: Log is configured to *not* fsync() on all Append() calls
I20260807 08:51:41.259680 4053 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0045c2909df44c0db614c7a1af15520c: No bootstrap required, opened a new log
I20260807 08:51:41.261180 4053 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0045c2909df44c0db614c7a1af15520c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0045c2909df44c0db614c7a1af15520c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43567 } }
I20260807 08:51:41.261303 4053 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0045c2909df44c0db614c7a1af15520c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:51:41.261330 4053 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0045c2909df44c0db614c7a1af15520c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0045c2909df44c0db614c7a1af15520c, State: Initialized, Role: FOLLOWER
I20260807 08:51:41.261422 4053 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0045c2909df44c0db614c7a1af15520c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0045c2909df44c0db614c7a1af15520c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43567 } }
I20260807 08:51:41.261471 4053 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0045c2909df44c0db614c7a1af15520c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:51:41.261488 4053 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0045c2909df44c0db614c7a1af15520c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:51:41.261518 4053 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0045c2909df44c0db614c7a1af15520c [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:51:41.261940 4053 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0045c2909df44c0db614c7a1af15520c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0045c2909df44c0db614c7a1af15520c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43567 } }
I20260807 08:51:41.262020 4053 leader_election.cc:302] T 00000000000000000000000000000000 P 0045c2909df44c0db614c7a1af15520c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0045c2909df44c0db614c7a1af15520c; no voters:
I20260807 08:51:41.262140 4053 leader_election.cc:288] T 00000000000000000000000000000000 P 0045c2909df44c0db614c7a1af15520c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:51:41.262181 4057 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0045c2909df44c0db614c7a1af15520c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:51:41.262369 4057 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0045c2909df44c0db614c7a1af15520c [term 1 LEADER]: Becoming Leader. State: Replica: 0045c2909df44c0db614c7a1af15520c, State: Running, Role: LEADER
I20260807 08:51:41.262368 4053 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0045c2909df44c0db614c7a1af15520c [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:51:41.262499 4057 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0045c2909df44c0db614c7a1af15520c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0045c2909df44c0db614c7a1af15520c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43567 } }
I20260807 08:51:41.262625 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 3950
I20260807 08:51:41.262734 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786092453486433-4734-0/minicluster-data/master-0/wal/instance
I20260807 08:51:41.262930 4734 external_mini_cluster.cc:1428] Setting key 2454a20adb6fd95fc140f4cede4773ee
I20260807 08:51:41.263733 4061 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0045c2909df44c0db614c7a1af15520c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0045c2909df44c0db614c7a1af15520c. Latest consensus state: current_term: 1 leader_uuid: "0045c2909df44c0db614c7a1af15520c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0045c2909df44c0db614c7a1af15520c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43567 } } }
I20260807 08:51:41.263855 4060 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0045c2909df44c0db614c7a1af15520c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0045c2909df44c0db614c7a1af15520c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0045c2909df44c0db614c7a1af15520c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43567 } } }
I20260807 08:51:41.264020 4060 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0045c2909df44c0db614c7a1af15520c [sys.catalog]: This master's current role is: LEADER
I20260807 08:51:41.264008 4061 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0045c2909df44c0db614c7a1af15520c [sys.catalog]: This master's current role is: LEADER
W20260807 08:51:41.264266 4081 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0045c2909df44c0db614c7a1af15520c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 08:51:41.264315 4081 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 08:51:41.265841 4086 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:51:41.266017 4086 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:51:41.267431 4086 catalog_manager.cc:1347] Generated new cluster ID: 434b3addfc2b42e3a2d28d79a2d141a8
I20260807 08:51:41.267531 4086 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 08:51:41 dist-test-slave-6zbq krb5kdc[3932](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092701, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 08:51:41.272846 4075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:41.263350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.181:52673 (local address 127.0.0.1:43567)
0807 08:51:41.263721 (+ 371us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:41.263729 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:41.263765 (+ 36us) server_negotiation.cc:407] Connection header received
0807 08:51:41.263968 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:41.263976 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:41.264045 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:41.264803 (+ 758us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:51:41.264872 (+ 69us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:41.265704 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:41.266776 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:41.267286 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:41.267423 (+ 137us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:41.269502 (+ 2079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:51:41.269506 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:51:41.269507 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:51:41.269517 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:51:41.271405 (+ 1888us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:51:41.271769 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:51:41.271771 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:51:41.271772 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:51:41.271862 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:51:41.272061 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:51:41.272062 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:51:41.272063 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:51:41.272302 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:51:41.272452 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:41.272563 (+ 111us) server_negotiation.cc:299] Negotiation successful
0807 08:51:41.272658 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327,"thread_start_us":76,"threads_started":1}
I20260807 08:51:41.273805 4734 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 08:51:41.277860 4086 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:51:41.278554 4086 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:51:41.283006 4075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:41.275023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.181:49903 (local address 127.0.0.1:43567)
0807 08:51:41.275055 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:41.275057 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:41.277678 (+ 2621us) server_negotiation.cc:407] Connection header received
0807 08:51:41.277700 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:41.277701 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:41.277710 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:41.277764 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:51:41.278281 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:41.278871 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:41.279536 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:41.279748 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:41.279800 (+ 52us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:41.280787 (+ 987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:51:41.280788 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:51:41.280788 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:51:41.280795 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:51:41.281955 (+ 1160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:51:41.282278 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:51:41.282279 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:51:41.282279 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:51:41.282327 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:51:41.282561 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:51:41.282562 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:51:41.282562 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:51:41.282658 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:51:41.282752 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:41.282872 (+ 120us) server_negotiation.cc:299] Negotiation successful
0807 08:51:41.282928 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":30,"server-negotiator.queue_time_us":16}
I20260807 08:51:41.283704 4086 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0045c2909df44c0db614c7a1af15520c: Generated new TSK 0
I20260807 08:51:41.283859 4086 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:51:41.286469 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 3950
W20260807 08:51:41.298108 3930 connection.cc:535] client connection to 127.0.0.1:43567 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43567): Connection reset by peer (error 104)
2026-08-07T08:51:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1381 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786092453486433-4734-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:51:41 dist-test-slave-6zbq krb5kdc[4097](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:51:41 dist-test-slave-6zbq krb5kdc[4097](info): set up 2 sockets
Aug 07 08:51:41 dist-test-slave-6zbq krb5kdc[4097](info): commencing operation
krb5kdc: starting...
W20260807 08:51:42.421993 4734 mini_kdc.cc:121] Time spent starting KDC: real 1.110s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786092453486433-4734-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786092453486433-4734-0/krb5kdc/test-user.keytab.
Aug 07 08:51:42 dist-test-slave-6zbq krb5kdc[4097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092702, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:51:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:51:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786092453486433-4734-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786092453486433-4734-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 08:51:42.528569 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786092453486433-4734-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786092453486433-4734-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786092453486433-4734-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38149
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786092453486433-4734-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38149
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:51:42.607728 4110 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:51:42.607983 4110 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:51:42.608016 4110 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:51:42.609470 4110 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:51:42.609521 4110 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:51:42.609542 4110 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:51:42.609565 4110 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:51:42.609588 4110 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:51:42.611619 4110 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786092453486433-4734-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786092453486433-4734-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38149
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786092453486433-4734-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38149
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4110
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786092453486433-4734-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:51:42.611876 4110 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:51:42.612111 4110 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:51:42.616532 4117 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:51:42.616545 4118 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:51:42.616696 4110 server_base.cc:1034] running on GCE node
W20260807 08:51:42.616554 4122 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:51:42.617028 4110 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:51:42.617259 4110 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:51:42.618389 4110 hybrid_clock.cc:648] HybridClock initialized: now 1786092702618374 us; error 28 us; skew 500 ppm
Aug 07 08:51:42 dist-test-slave-6zbq krb5kdc[4097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092702, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:51:42.620566 4110 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 08:51:42.620947 4110 webserver.cc:460] Webserver started at http://127.0.0.1:39779/ using document root <none> and password file <none>
I20260807 08:51:42.621134 4110 fs_manager.cc:359] Metadata directory not provided
I20260807 08:51:42.621173 4110 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:51:42.621259 4110 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:51:42.622255 4110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786092453486433-4734-0/minicluster-data/master-0/data/instance:
uuid: "732f2874689d494db18b7a127a9d2d24"
format_stamp: "Formatted at 2026-08-07 08:51:42 on dist-test-slave-6zbq"
server_key: "c61e4dbedd3781aa964ea55769e2f780"
server_key_iv: "03425090b6096a5b9e2fedbffa7f91fb"
server_key_version: "encryptionkey@0"
I20260807 08:51:42.622597 4110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786092453486433-4734-0/minicluster-data/master-0/wal/instance:
uuid: "732f2874689d494db18b7a127a9d2d24"
format_stamp: "Formatted at 2026-08-07 08:51:42 on dist-test-slave-6zbq"
server_key: "c61e4dbedd3781aa964ea55769e2f780"
server_key_iv: "03425090b6096a5b9e2fedbffa7f91fb"
server_key_version: "encryptionkey@0"
I20260807 08:51:42.623826 4110 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:51:42.624439 4132 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:51:42.624580 4110 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 08:51:42.624656 4110 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786092453486433-4734-0/minicluster-data/master-0/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786092453486433-4734-0/minicluster-data/master-0/wal
uuid: "732f2874689d494db18b7a127a9d2d24"
format_stamp: "Formatted at 2026-08-07 08:51:42 on dist-test-slave-6zbq"
server_key: "c61e4dbedd3781aa964ea55769e2f780"
server_key_iv: "03425090b6096a5b9e2fedbffa7f91fb"
server_key_version: "encryptionkey@0"
I20260807 08:51:42.624717 4110 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786092453486433-4734-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786092453486433-4734-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786092453486433-4734-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:51:42.648334 4110 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:51:42.650086 4110 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:51:42.650195 4110 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:51:42.653963 4110 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38149
I20260807 08:51:42.654012 4194 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38149 every 8 connection(s)
I20260807 08:51:42.654318 4110 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
I20260807 08:51:42.655228 4196 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:51:42.657377 4196 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 732f2874689d494db18b7a127a9d2d24: Bootstrap starting.
I20260807 08:51:42.657946 4196 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 732f2874689d494db18b7a127a9d2d24: Neither blocks nor log segments found. Creating new log.
I20260807 08:51:42.658201 4196 log.cc:824] T 00000000000000000000000000000000 P 732f2874689d494db18b7a127a9d2d24: Log is configured to *not* fsync() on all Append() calls
I20260807 08:51:42.658849 4196 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 732f2874689d494db18b7a127a9d2d24: No bootstrap required, opened a new log
I20260807 08:51:42.660382 4196 raft_consensus.cc:348] T 00000000000000000000000000000000 P 732f2874689d494db18b7a127a9d2d24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "732f2874689d494db18b7a127a9d2d24" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38149 } }
I20260807 08:51:42.660516 4196 raft_consensus.cc:374] T 00000000000000000000000000000000 P 732f2874689d494db18b7a127a9d2d24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:51:42.660549 4196 raft_consensus.cc:729] T 00000000000000000000000000000000 P 732f2874689d494db18b7a127a9d2d24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 732f2874689d494db18b7a127a9d2d24, State: Initialized, Role: FOLLOWER
I20260807 08:51:42.660671 4196 consensus_queue.cc:260] T 00000000000000000000000000000000 P 732f2874689d494db18b7a127a9d2d24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "732f2874689d494db18b7a127a9d2d24" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38149 } }
I20260807 08:51:42.660732 4196 raft_consensus.cc:388] T 00000000000000000000000000000000 P 732f2874689d494db18b7a127a9d2d24 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:51:42.660770 4196 raft_consensus.cc:482] T 00000000000000000000000000000000 P 732f2874689d494db18b7a127a9d2d24 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:51:42.660817 4196 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 732f2874689d494db18b7a127a9d2d24 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:51:42.661314 4196 raft_consensus.cc:504] T 00000000000000000000000000000000 P 732f2874689d494db18b7a127a9d2d24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "732f2874689d494db18b7a127a9d2d24" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38149 } }
I20260807 08:51:42.661407 4196 leader_election.cc:302] T 00000000000000000000000000000000 P 732f2874689d494db18b7a127a9d2d24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 732f2874689d494db18b7a127a9d2d24; no voters:
I20260807 08:51:42.661552 4196 leader_election.cc:288] T 00000000000000000000000000000000 P 732f2874689d494db18b7a127a9d2d24 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:51:42.661578 4204 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 732f2874689d494db18b7a127a9d2d24 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:51:42.661703 4204 raft_consensus.cc:686] T 00000000000000000000000000000000 P 732f2874689d494db18b7a127a9d2d24 [term 1 LEADER]: Becoming Leader. State: Replica: 732f2874689d494db18b7a127a9d2d24, State: Running, Role: LEADER
I20260807 08:51:42.661793 4196 sys_catalog.cc:564] T 00000000000000000000000000000000 P 732f2874689d494db18b7a127a9d2d24 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:51:42.661834 4204 consensus_queue.cc:237] T 00000000000000000000000000000000 P 732f2874689d494db18b7a127a9d2d24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "732f2874689d494db18b7a127a9d2d24" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38149 } }
I20260807 08:51:42.662184 4207 sys_catalog.cc:455] T 00000000000000000000000000000000 P 732f2874689d494db18b7a127a9d2d24 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 732f2874689d494db18b7a127a9d2d24. Latest consensus state: current_term: 1 leader_uuid: "732f2874689d494db18b7a127a9d2d24" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "732f2874689d494db18b7a127a9d2d24" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38149 } } }
I20260807 08:51:42.662307 4205 sys_catalog.cc:455] T 00000000000000000000000000000000 P 732f2874689d494db18b7a127a9d2d24 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "732f2874689d494db18b7a127a9d2d24" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "732f2874689d494db18b7a127a9d2d24" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38149 } } }
I20260807 08:51:42.662518 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 4110
I20260807 08:51:42.662624 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786092453486433-4734-0/minicluster-data/master-0/wal/instance
I20260807 08:51:42.662842 4734 external_mini_cluster.cc:1428] Setting key ec346794f71dab80bc648f7d43c8ddaa
I20260807 08:51:42.662418 4205 sys_catalog.cc:458] T 00000000000000000000000000000000 P 732f2874689d494db18b7a127a9d2d24 [sys.catalog]: This master's current role is: LEADER
I20260807 08:51:42.662315 4207 sys_catalog.cc:458] T 00000000000000000000000000000000 P 732f2874689d494db18b7a127a9d2d24 [sys.catalog]: This master's current role is: LEADER
W20260807 08:51:42.663564 4235 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 732f2874689d494db18b7a127a9d2d24: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 08:51:42.663609 4235 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 08:51:42.663998 4239 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:51:42.664171 4239 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:51:42.665598 4239 catalog_manager.cc:1347] Generated new cluster ID: 21668255438442e39f1d79d3894295d5
I20260807 08:51:42.665645 4239 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:51:42.668349 4243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:42.663391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.181:42279 (local address 127.0.0.1:38149)
0807 08:51:42.663574 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:42.663582 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:42.663612 (+ 30us) server_negotiation.cc:407] Connection header received
0807 08:51:42.663721 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:42.663728 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:42.663783 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:42.663912 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:51:42.664144 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:42.665289 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:42.666390 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:42.666975 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:42.667065 (+ 90us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:42.667725 (+ 660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:51:42.667729 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:51:42.667729 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:51:42.667915 (+ 186us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:51:42.667949 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:51:42.667963 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:51:42.668038 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:42.668127 (+ 89us) server_negotiation.cc:299] Negotiation successful
0807 08:51:42.668150 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":57,"threads_started":1}
I20260807 08:51:42.669409 4734 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 08:51:42.673323 4239 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:51:42.673913 4239 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:51:42.674233 4243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:42.670862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.181:46471 (local address 127.0.0.1:38149)
0807 08:51:42.670922 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:42.670924 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:42.671005 (+ 81us) server_negotiation.cc:407] Connection header received
0807 08:51:42.671079 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:42.671080 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:42.671088 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:42.671115 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:51:42.671297 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:42.671992 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:42.672975 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:42.673208 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:42.673259 (+ 51us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:42.673992 (+ 733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:51:42.673993 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:51:42.673993 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:51:42.673997 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:51:42.674010 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:51:42.674014 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:51:42.674045 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:42.674158 (+ 113us) server_negotiation.cc:299] Negotiation successful
0807 08:51:42.674169 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 08:51:42.682749 4239 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 732f2874689d494db18b7a127a9d2d24: Generated new TSK 0
I20260807 08:51:42.682904 4239 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:51:42.684805 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 4110
W20260807 08:51:42.697036 4095 connection.cc:535] client connection to 127.0.0.1:38149 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38149): Connection reset by peer (error 104)
2026-08-07T08:51:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1398 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786092453486433-4734-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:51:42 dist-test-slave-6zbq krb5kdc[4253](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:51:42 dist-test-slave-6zbq krb5kdc[4253](info): set up 2 sockets
Aug 07 08:51:42 dist-test-slave-6zbq krb5kdc[4253](info): commencing operation
krb5kdc: starting...
W20260807 08:51:43.798050 4734 mini_kdc.cc:121] Time spent starting KDC: real 1.088s user 0.005s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786092453486433-4734-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786092453486433-4734-0/krb5kdc/test-user.keytab.
Aug 07 08:51:43 dist-test-slave-6zbq krb5kdc[4253](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092703, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:51:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:51:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786092453486433-4734-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786092453486433-4734-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 08:51:43.905759 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786092453486433-4734-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786092453486433-4734-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786092453486433-4734-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40259
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786092453486433-4734-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40259
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:51:43.983814 4270 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:51:43.983995 4270 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:51:43.984017 4270 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:51:43.985086 4270 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:51:43.985121 4270 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:51:43.985133 4270 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:51:43.985145 4270 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:51:43.985165 4270 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:51:43.986600 4270 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786092453486433-4734-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786092453486433-4734-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40259
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786092453486433-4734-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40259
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4270
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786092453486433-4734-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:51:43.986790 4270 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:51:43.986955 4270 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:51:43.991475 4280 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:51:43.991601 4270 server_base.cc:1034] running on GCE node
W20260807 08:51:43.991488 4277 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:51:43.991487 4278 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:51:43.991914 4270 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:51:43.992167 4270 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:51:43.993301 4270 hybrid_clock.cc:648] HybridClock initialized: now 1786092703993290 us; error 27 us; skew 500 ppm
Aug 07 08:51:43 dist-test-slave-6zbq krb5kdc[4253](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092703, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:51:43.995515 4270 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 08:51:43.995887 4270 webserver.cc:460] Webserver started at http://127.0.0.1:44775/ using document root <none> and password file <none>
I20260807 08:51:43.996070 4270 fs_manager.cc:359] Metadata directory not provided
I20260807 08:51:43.996116 4270 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:51:43.996234 4270 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:51:43.997234 4270 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786092453486433-4734-0/minicluster-data/master-0/data/instance:
uuid: "a99d5b76b0bc43b4a436d5f4f87a8c47"
format_stamp: "Formatted at 2026-08-07 08:51:43 on dist-test-slave-6zbq"
server_key: "6f9204fa98fc0eb7917148f0f5286b71"
server_key_iv: "1c4e918139df3b08df0f4b4d1bd0ab56"
server_key_version: "encryptionkey@0"
I20260807 08:51:43.997543 4270 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786092453486433-4734-0/minicluster-data/master-0/wal/instance:
uuid: "a99d5b76b0bc43b4a436d5f4f87a8c47"
format_stamp: "Formatted at 2026-08-07 08:51:43 on dist-test-slave-6zbq"
server_key: "6f9204fa98fc0eb7917148f0f5286b71"
server_key_iv: "1c4e918139df3b08df0f4b4d1bd0ab56"
server_key_version: "encryptionkey@0"
I20260807 08:51:43.998755 4270 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260807 08:51:43.999389 4289 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:51:43.999532 4270 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 08:51:43.999612 4270 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786092453486433-4734-0/minicluster-data/master-0/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786092453486433-4734-0/minicluster-data/master-0/wal
uuid: "a99d5b76b0bc43b4a436d5f4f87a8c47"
format_stamp: "Formatted at 2026-08-07 08:51:43 on dist-test-slave-6zbq"
server_key: "6f9204fa98fc0eb7917148f0f5286b71"
server_key_iv: "1c4e918139df3b08df0f4b4d1bd0ab56"
server_key_version: "encryptionkey@0"
I20260807 08:51:43.999672 4270 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786092453486433-4734-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786092453486433-4734-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786092453486433-4734-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:51:44.002718 4270 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:51:44.004266 4270 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:51:44.004380 4270 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:51:44.007977 4270 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40259
I20260807 08:51:44.008008 4361 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40259 every 8 connection(s)
I20260807 08:51:44.008268 4270 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
I20260807 08:51:44.009109 4362 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:51:44.009295 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 4270
I20260807 08:51:44.009395 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786092453486433-4734-0/minicluster-data/master-0/wal/instance
I20260807 08:51:44.009549 4734 external_mini_cluster.cc:1428] Setting key 45b82ed0b2d6249dbb5b62dadf02415b
I20260807 08:51:44.011742 4366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:44.009978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.181:35007 (local address 127.0.0.1:40259)
0807 08:51:44.010162 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:44.010171 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:44.010208 (+ 37us) server_negotiation.cc:407] Connection header received
0807 08:51:44.010647 (+ 439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:44.010656 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:44.010690 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:44.010845 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:51:44.011355 (+ 510us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:51:44.011358 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:51:44.011358 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:51:44.011363 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:51:44.011403 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:51:44.011417 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:51:44.011491 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:44.011529 (+ 38us) server_negotiation.cc:299] Negotiation successful
0807 08:51:44.011569 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":74,"threads_started":1}
I20260807 08:51:44.012259 4362 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a99d5b76b0bc43b4a436d5f4f87a8c47: Bootstrap starting.
I20260807 08:51:44.012624 4734 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 08:51:44.012883 4362 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a99d5b76b0bc43b4a436d5f4f87a8c47: Neither blocks nor log segments found. Creating new log.
I20260807 08:51:44.013127 4362 log.cc:824] T 00000000000000000000000000000000 P a99d5b76b0bc43b4a436d5f4f87a8c47: Log is configured to *not* fsync() on all Append() calls
I20260807 08:51:44.013847 4362 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a99d5b76b0bc43b4a436d5f4f87a8c47: No bootstrap required, opened a new log
I20260807 08:51:44.014689 4366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:44.013829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.181:53729 (local address 127.0.0.1:40259)
0807 08:51:44.013863 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:44.013865 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:44.013942 (+ 77us) server_negotiation.cc:407] Connection header received
0807 08:51:44.014003 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:44.014003 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:44.014007 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:44.014049 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:51:44.014519 (+ 470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:51:44.014520 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:51:44.014521 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:51:44.014524 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:51:44.014534 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:51:44.014538 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:51:44.014576 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:44.014627 (+ 51us) server_negotiation.cc:299] Negotiation successful
0807 08:51:44.014637 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 08:51:44.015760 4362 raft_consensus.cc:348] T 00000000000000000000000000000000 P a99d5b76b0bc43b4a436d5f4f87a8c47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a99d5b76b0bc43b4a436d5f4f87a8c47" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40259 } }
I20260807 08:51:44.015883 4362 raft_consensus.cc:374] T 00000000000000000000000000000000 P a99d5b76b0bc43b4a436d5f4f87a8c47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:51:44.015904 4362 raft_consensus.cc:729] T 00000000000000000000000000000000 P a99d5b76b0bc43b4a436d5f4f87a8c47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a99d5b76b0bc43b4a436d5f4f87a8c47, State: Initialized, Role: FOLLOWER
I20260807 08:51:44.015985 4362 consensus_queue.cc:260] T 00000000000000000000000000000000 P a99d5b76b0bc43b4a436d5f4f87a8c47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a99d5b76b0bc43b4a436d5f4f87a8c47" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40259 } }
I20260807 08:51:44.016057 4362 raft_consensus.cc:388] T 00000000000000000000000000000000 P a99d5b76b0bc43b4a436d5f4f87a8c47 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:51:44.016084 4362 raft_consensus.cc:482] T 00000000000000000000000000000000 P a99d5b76b0bc43b4a436d5f4f87a8c47 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:51:44.016114 4362 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a99d5b76b0bc43b4a436d5f4f87a8c47 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:51:44.016553 4362 raft_consensus.cc:504] T 00000000000000000000000000000000 P a99d5b76b0bc43b4a436d5f4f87a8c47 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a99d5b76b0bc43b4a436d5f4f87a8c47" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40259 } }
I20260807 08:51:44.016634 4362 leader_election.cc:302] T 00000000000000000000000000000000 P a99d5b76b0bc43b4a436d5f4f87a8c47 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a99d5b76b0bc43b4a436d5f4f87a8c47; no voters:
I20260807 08:51:44.016744 4362 leader_election.cc:288] T 00000000000000000000000000000000 P a99d5b76b0bc43b4a436d5f4f87a8c47 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:51:44.016796 4377 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a99d5b76b0bc43b4a436d5f4f87a8c47 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:51:44.016945 4362 sys_catalog.cc:564] T 00000000000000000000000000000000 P a99d5b76b0bc43b4a436d5f4f87a8c47 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:51:44.016970 4377 raft_consensus.cc:686] T 00000000000000000000000000000000 P a99d5b76b0bc43b4a436d5f4f87a8c47 [term 1 LEADER]: Becoming Leader. State: Replica: a99d5b76b0bc43b4a436d5f4f87a8c47, State: Running, Role: LEADER
I20260807 08:51:44.017109 4377 consensus_queue.cc:237] T 00000000000000000000000000000000 P a99d5b76b0bc43b4a436d5f4f87a8c47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a99d5b76b0bc43b4a436d5f4f87a8c47" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40259 } }
I20260807 08:51:44.017454 4378 sys_catalog.cc:455] T 00000000000000000000000000000000 P a99d5b76b0bc43b4a436d5f4f87a8c47 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a99d5b76b0bc43b4a436d5f4f87a8c47" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a99d5b76b0bc43b4a436d5f4f87a8c47" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40259 } } }
I20260807 08:51:44.017540 4378 sys_catalog.cc:458] T 00000000000000000000000000000000 P a99d5b76b0bc43b4a436d5f4f87a8c47 [sys.catalog]: This master's current role is: LEADER
I20260807 08:51:44.017735 4393 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:51:44.017750 4379 sys_catalog.cc:455] T 00000000000000000000000000000000 P a99d5b76b0bc43b4a436d5f4f87a8c47 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a99d5b76b0bc43b4a436d5f4f87a8c47. Latest consensus state: current_term: 1 leader_uuid: "a99d5b76b0bc43b4a436d5f4f87a8c47" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a99d5b76b0bc43b4a436d5f4f87a8c47" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40259 } } }
I20260807 08:51:44.018020 4379 sys_catalog.cc:458] T 00000000000000000000000000000000 P a99d5b76b0bc43b4a436d5f4f87a8c47 [sys.catalog]: This master's current role is: LEADER
I20260807 08:51:44.018445 4393 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:51:44.019814 4393 catalog_manager.cc:1347] Generated new cluster ID: f47749766c5c4be8b8ad854b2cb505f9
I20260807 08:51:44.019862 4393 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:51:44.039860 4393 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:51:44.039995 4393 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:51:44.045231 4393 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a99d5b76b0bc43b4a436d5f4f87a8c47: Generated new TSK 0
I20260807 08:51:44.045382 4393 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:51:44.049773 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 4270
2026-08-07T08:51:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1361 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786092453486433-4734-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:51:44 dist-test-slave-6zbq krb5kdc[4414](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:51:44 dist-test-slave-6zbq krb5kdc[4414](info): set up 2 sockets
Aug 07 08:51:44 dist-test-slave-6zbq krb5kdc[4414](info): commencing operation
krb5kdc: starting...
W20260807 08:51:45.167032 4734 mini_kdc.cc:121] Time spent starting KDC: real 1.095s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786092453486433-4734-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786092453486433-4734-0/krb5kdc/test-user.keytab.
Aug 07 08:51:45 dist-test-slave-6zbq krb5kdc[4414](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092705, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:51:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:51:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786092453486433-4734-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786092453486433-4734-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 08:51:45.281069 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786092453486433-4734-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786092453486433-4734-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786092453486433-4734-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43417
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786092453486433-4734-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43417
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:51:45.359824 4434 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:51:45.360008 4434 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:51:45.360038 4434 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:51:45.361150 4434 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:51:45.361188 4434 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:51:45.361207 4434 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:51:45.361223 4434 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:51:45.361239 4434 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:51:45.362694 4434 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786092453486433-4734-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786092453486433-4734-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43417
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786092453486433-4734-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43417
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4434
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786092453486433-4734-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:51:45.362919 4434 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:51:45.363114 4434 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:51:45.367424 4439 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:51:45.367432 4440 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:51:45.367422 4443 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:51:45.367661 4434 server_base.cc:1034] running on GCE node
I20260807 08:51:45.367895 4434 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:51:45.368147 4434 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:51:45.369285 4434 hybrid_clock.cc:648] HybridClock initialized: now 1786092705369263 us; error 34 us; skew 500 ppm
Aug 07 08:51:45 dist-test-slave-6zbq krb5kdc[4414](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092705, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:51:45.374841 4434 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 08:51:45.375219 4434 webserver.cc:460] Webserver started at http://127.0.0.1:45285/ using document root <none> and password file <none>
I20260807 08:51:45.375427 4434 fs_manager.cc:359] Metadata directory not provided
I20260807 08:51:45.375468 4434 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:51:45.375581 4434 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:51:45.376559 4434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786092453486433-4734-0/minicluster-data/master-0/data/instance:
uuid: "2e17bb4b6ae24166b6502b7cabab3232"
format_stamp: "Formatted at 2026-08-07 08:51:45 on dist-test-slave-6zbq"
server_key: "6f36249aace8e0f7607558f3b6a166c4"
server_key_iv: "7af21adc21f46e567e5ed81004af7190"
server_key_version: "encryptionkey@0"
I20260807 08:51:45.376878 4434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786092453486433-4734-0/minicluster-data/master-0/wal/instance:
uuid: "2e17bb4b6ae24166b6502b7cabab3232"
format_stamp: "Formatted at 2026-08-07 08:51:45 on dist-test-slave-6zbq"
server_key: "6f36249aace8e0f7607558f3b6a166c4"
server_key_iv: "7af21adc21f46e567e5ed81004af7190"
server_key_version: "encryptionkey@0"
I20260807 08:51:45.378039 4434 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260807 08:51:45.378722 4453 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:51:45.378868 4434 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 08:51:45.378948 4434 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786092453486433-4734-0/minicluster-data/master-0/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786092453486433-4734-0/minicluster-data/master-0/wal
uuid: "2e17bb4b6ae24166b6502b7cabab3232"
format_stamp: "Formatted at 2026-08-07 08:51:45 on dist-test-slave-6zbq"
server_key: "6f36249aace8e0f7607558f3b6a166c4"
server_key_iv: "7af21adc21f46e567e5ed81004af7190"
server_key_version: "encryptionkey@0"
I20260807 08:51:45.379014 4434 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786092453486433-4734-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786092453486433-4734-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786092453486433-4734-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:51:45.389771 4434 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:51:45.391434 4434 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:51:45.391552 4434 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:51:45.395236 4434 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43417
I20260807 08:51:45.395270 4528 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43417 every 8 connection(s)
I20260807 08:51:45.395555 4434 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
I20260807 08:51:45.396441 4529 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:51:45.398584 4529 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e17bb4b6ae24166b6502b7cabab3232: Bootstrap starting.
I20260807 08:51:45.399078 4529 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2e17bb4b6ae24166b6502b7cabab3232: Neither blocks nor log segments found. Creating new log.
I20260807 08:51:45.399308 4529 log.cc:824] T 00000000000000000000000000000000 P 2e17bb4b6ae24166b6502b7cabab3232: Log is configured to *not* fsync() on all Append() calls
I20260807 08:51:45.399888 4529 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e17bb4b6ae24166b6502b7cabab3232: No bootstrap required, opened a new log
I20260807 08:51:45.401396 4529 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2e17bb4b6ae24166b6502b7cabab3232 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e17bb4b6ae24166b6502b7cabab3232" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43417 } }
I20260807 08:51:45.401521 4529 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2e17bb4b6ae24166b6502b7cabab3232 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:51:45.401548 4529 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2e17bb4b6ae24166b6502b7cabab3232 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e17bb4b6ae24166b6502b7cabab3232, State: Initialized, Role: FOLLOWER
I20260807 08:51:45.401648 4529 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2e17bb4b6ae24166b6502b7cabab3232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e17bb4b6ae24166b6502b7cabab3232" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43417 } }
I20260807 08:51:45.401723 4529 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2e17bb4b6ae24166b6502b7cabab3232 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:51:45.401751 4529 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2e17bb4b6ae24166b6502b7cabab3232 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:51:45.401782 4529 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2e17bb4b6ae24166b6502b7cabab3232 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:51:45.402216 4529 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2e17bb4b6ae24166b6502b7cabab3232 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e17bb4b6ae24166b6502b7cabab3232" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43417 } }
I20260807 08:51:45.402302 4529 leader_election.cc:302] T 00000000000000000000000000000000 P 2e17bb4b6ae24166b6502b7cabab3232 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2e17bb4b6ae24166b6502b7cabab3232; no voters:
I20260807 08:51:45.402453 4529 leader_election.cc:288] T 00000000000000000000000000000000 P 2e17bb4b6ae24166b6502b7cabab3232 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:51:45.402490 4534 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2e17bb4b6ae24166b6502b7cabab3232 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:51:45.402635 4534 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2e17bb4b6ae24166b6502b7cabab3232 [term 1 LEADER]: Becoming Leader. State: Replica: 2e17bb4b6ae24166b6502b7cabab3232, State: Running, Role: LEADER
I20260807 08:51:45.402674 4529 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2e17bb4b6ae24166b6502b7cabab3232 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:51:45.402724 4534 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2e17bb4b6ae24166b6502b7cabab3232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e17bb4b6ae24166b6502b7cabab3232" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43417 } }
I20260807 08:51:45.403070 4537 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e17bb4b6ae24166b6502b7cabab3232 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2e17bb4b6ae24166b6502b7cabab3232. Latest consensus state: current_term: 1 leader_uuid: "2e17bb4b6ae24166b6502b7cabab3232" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e17bb4b6ae24166b6502b7cabab3232" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43417 } } }
I20260807 08:51:45.403182 4536 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e17bb4b6ae24166b6502b7cabab3232 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2e17bb4b6ae24166b6502b7cabab3232" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e17bb4b6ae24166b6502b7cabab3232" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43417 } } }
I20260807 08:51:45.403206 4537 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e17bb4b6ae24166b6502b7cabab3232 [sys.catalog]: This master's current role is: LEADER
I20260807 08:51:45.403280 4536 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e17bb4b6ae24166b6502b7cabab3232 [sys.catalog]: This master's current role is: LEADER
I20260807 08:51:45.403587 4546 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:51:45.404063 4546 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:51:45.405061 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 4434
I20260807 08:51:45.405179 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786092453486433-4734-0/minicluster-data/master-0/wal/instance
I20260807 08:51:45.405368 4734 external_mini_cluster.cc:1428] Setting key 451c0eb086c2cadd4a5f72d99c8b4cee
I20260807 08:51:45.406107 4546 catalog_manager.cc:1347] Generated new cluster ID: 59766906f7464c83ae3b1fc38844822a
I20260807 08:51:45.406152 4546 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 08:51:45 dist-test-slave-6zbq krb5kdc[4414](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092705, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 08:51:45.420568 4561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:45.405894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.181:32875 (local address 127.0.0.1:43417)
0807 08:51:45.406029 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:45.406038 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:45.410544 (+ 4506us) server_negotiation.cc:407] Connection header received
0807 08:51:45.410668 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:45.410675 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:45.410730 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:45.410884 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:51:45.411533 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:45.412975 (+ 1442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:45.413951 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:45.414587 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:45.414699 (+ 112us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:45.417269 (+ 2570us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:51:45.417273 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:51:45.417273 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:51:45.417284 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:51:45.419030 (+ 1746us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:51:45.419442 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:51:45.419445 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:51:45.419445 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:51:45.419538 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:51:45.419739 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:51:45.419739 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:51:45.419739 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:51:45.420011 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:51:45.420163 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:45.420274 (+ 111us) server_negotiation.cc:299] Negotiation successful
0807 08:51:45.420365 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":60,"threads_started":1}
I20260807 08:51:45.421438 4734 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 08:51:45.426635 4546 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:51:45.427186 4546 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:51:45.433051 4546 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2e17bb4b6ae24166b6502b7cabab3232: Generated new TSK 0
I20260807 08:51:45.433240 4546 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:51:45.433501 4561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:45.426582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.181:40065 (local address 127.0.0.1:43417)
0807 08:51:45.426614 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:45.426616 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:45.426810 (+ 194us) server_negotiation.cc:407] Connection header received
0807 08:51:45.426885 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:45.426886 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:45.426895 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:45.426941 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:51:45.427530 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:45.428249 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:45.429201 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:45.429500 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:45.429549 (+ 49us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:45.430827 (+ 1278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:51:45.430828 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:51:45.430828 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:51:45.430835 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:51:45.432228 (+ 1393us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:51:45.432634 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:51:45.432635 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:51:45.432635 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:51:45.432694 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:51:45.432926 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:51:45.432927 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:51:45.432928 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:51:45.433058 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:51:45.433157 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:45.433386 (+ 229us) server_negotiation.cc:299] Negotiation successful
0807 08:51:45.433450 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260807 08:51:45.435241 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 4434
W20260807 08:51:45.449702 4412 connection.cc:535] client connection to 127.0.0.1:43417 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43417): Connection reset by peer (error 104)
2026-08-07T08:51:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1390 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786092453486433-4734-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:51:45 dist-test-slave-6zbq krb5kdc[4578](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:51:45 dist-test-slave-6zbq krb5kdc[4578](info): set up 2 sockets
Aug 07 08:51:45 dist-test-slave-6zbq krb5kdc[4578](info): commencing operation
krb5kdc: starting...
W20260807 08:51:46.558147 4734 mini_kdc.cc:121] Time spent starting KDC: real 1.095s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786092453486433-4734-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786092453486433-4734-0/krb5kdc/test-user.keytab.
Aug 07 08:51:46 dist-test-slave-6zbq krb5kdc[4578](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092706, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:51:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:51:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786092453486433-4734-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786092453486433-4734-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 08:51:46.666385 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786092453486433-4734-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786092453486433-4734-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786092453486433-4734-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33809
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786092453486433-4734-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33809
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:51:46.746807 4600 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:51:46.746982 4600 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:51:46.747004 4600 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:51:46.748098 4600 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:51:46.748138 4600 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:51:46.748152 4600 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:51:46.748175 4600 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:51:46.748198 4600 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:51:46.749619 4600 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786092453486433-4734-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786092453486433-4734-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33809
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786092453486433-4734-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33809
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4600
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786092453486433-4734-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:51:46.749835 4600 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:51:46.750018 4600 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:51:46.754384 4609 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:51:46.754402 4607 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:51:46.754402 4613 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:51:46.754442 4600 server_base.cc:1034] running on GCE node
I20260807 08:51:46.754747 4600 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:51:46.755005 4600 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:51:46.756138 4600 hybrid_clock.cc:648] HybridClock initialized: now 1786092706756118 us; error 33 us; skew 500 ppm
Aug 07 08:51:46 dist-test-slave-6zbq krb5kdc[4578](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092706, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:51:46.758415 4600 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 08:51:46.758828 4600 webserver.cc:460] Webserver started at http://127.0.0.1:38125/ using document root <none> and password file <none>
I20260807 08:51:46.759017 4600 fs_manager.cc:359] Metadata directory not provided
I20260807 08:51:46.759069 4600 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:51:46.759188 4600 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:51:46.760109 4600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786092453486433-4734-0/minicluster-data/master-0/data/instance:
uuid: "e860dbdb17ce4d93882dcc337cd71bcf"
format_stamp: "Formatted at 2026-08-07 08:51:46 on dist-test-slave-6zbq"
server_key: "c04e42ebf169c8b63662a36d30183fa5"
server_key_iv: "8d9640b3263a56885bb177a6bac7b1ee"
server_key_version: "encryptionkey@0"
I20260807 08:51:46.760430 4600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786092453486433-4734-0/minicluster-data/master-0/wal/instance:
uuid: "e860dbdb17ce4d93882dcc337cd71bcf"
format_stamp: "Formatted at 2026-08-07 08:51:46 on dist-test-slave-6zbq"
server_key: "c04e42ebf169c8b63662a36d30183fa5"
server_key_iv: "8d9640b3263a56885bb177a6bac7b1ee"
server_key_version: "encryptionkey@0"
I20260807 08:51:46.761608 4600 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 08:51:46.762239 4623 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:51:46.762413 4600 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 08:51:46.762493 4600 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786092453486433-4734-0/minicluster-data/master-0/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786092453486433-4734-0/minicluster-data/master-0/wal
uuid: "e860dbdb17ce4d93882dcc337cd71bcf"
format_stamp: "Formatted at 2026-08-07 08:51:46 on dist-test-slave-6zbq"
server_key: "c04e42ebf169c8b63662a36d30183fa5"
server_key_iv: "8d9640b3263a56885bb177a6bac7b1ee"
server_key_version: "encryptionkey@0"
I20260807 08:51:46.762557 4600 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786092453486433-4734-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786092453486433-4734-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786092453486433-4734-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:51:46.779798 4600 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:51:46.781679 4600 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:51:46.781806 4600 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:51:46.785522 4600 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33809
I20260807 08:51:46.785552 4698 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33809 every 8 connection(s)
I20260807 08:51:46.785833 4600 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
I20260807 08:51:46.786580 4700 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:51:46.788580 4700 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e860dbdb17ce4d93882dcc337cd71bcf: Bootstrap starting.
I20260807 08:51:46.789090 4700 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e860dbdb17ce4d93882dcc337cd71bcf: Neither blocks nor log segments found. Creating new log.
I20260807 08:51:46.789343 4700 log.cc:824] T 00000000000000000000000000000000 P e860dbdb17ce4d93882dcc337cd71bcf: Log is configured to *not* fsync() on all Append() calls
I20260807 08:51:46.789939 4700 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e860dbdb17ce4d93882dcc337cd71bcf: No bootstrap required, opened a new log
I20260807 08:51:46.790160 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 4600
I20260807 08:51:46.790273 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786092453486433-4734-0/minicluster-data/master-0/wal/instance
I20260807 08:51:46.790477 4734 external_mini_cluster.cc:1428] Setting key ea6468c1db43e29c1c4889471a32158f
I20260807 08:51:46.791905 4700 raft_consensus.cc:348] T 00000000000000000000000000000000 P e860dbdb17ce4d93882dcc337cd71bcf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e860dbdb17ce4d93882dcc337cd71bcf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33809 } }
I20260807 08:51:46.792061 4700 raft_consensus.cc:374] T 00000000000000000000000000000000 P e860dbdb17ce4d93882dcc337cd71bcf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:51:46.792135 4700 raft_consensus.cc:729] T 00000000000000000000000000000000 P e860dbdb17ce4d93882dcc337cd71bcf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e860dbdb17ce4d93882dcc337cd71bcf, State: Initialized, Role: FOLLOWER
I20260807 08:51:46.792266 4700 consensus_queue.cc:260] T 00000000000000000000000000000000 P e860dbdb17ce4d93882dcc337cd71bcf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e860dbdb17ce4d93882dcc337cd71bcf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33809 } }
I20260807 08:51:46.792335 4700 raft_consensus.cc:388] T 00000000000000000000000000000000 P e860dbdb17ce4d93882dcc337cd71bcf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:51:46.792366 4700 raft_consensus.cc:482] T 00000000000000000000000000000000 P e860dbdb17ce4d93882dcc337cd71bcf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:51:46.792410 4700 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e860dbdb17ce4d93882dcc337cd71bcf [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:51:46.792892 4700 raft_consensus.cc:504] T 00000000000000000000000000000000 P e860dbdb17ce4d93882dcc337cd71bcf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e860dbdb17ce4d93882dcc337cd71bcf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33809 } }
I20260807 08:51:46.792984 4700 leader_election.cc:302] T 00000000000000000000000000000000 P e860dbdb17ce4d93882dcc337cd71bcf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e860dbdb17ce4d93882dcc337cd71bcf; no voters:
I20260807 08:51:46.793130 4700 leader_election.cc:288] T 00000000000000000000000000000000 P e860dbdb17ce4d93882dcc337cd71bcf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:51:46.793161 4706 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e860dbdb17ce4d93882dcc337cd71bcf [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:51:46.793260 4706 raft_consensus.cc:686] T 00000000000000000000000000000000 P e860dbdb17ce4d93882dcc337cd71bcf [term 1 LEADER]: Becoming Leader. State: Replica: e860dbdb17ce4d93882dcc337cd71bcf, State: Running, Role: LEADER
I20260807 08:51:46.793376 4700 sys_catalog.cc:564] T 00000000000000000000000000000000 P e860dbdb17ce4d93882dcc337cd71bcf [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:51:46.793391 4706 consensus_queue.cc:237] T 00000000000000000000000000000000 P e860dbdb17ce4d93882dcc337cd71bcf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e860dbdb17ce4d93882dcc337cd71bcf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33809 } }
I20260807 08:51:46.793720 4706 sys_catalog.cc:455] T 00000000000000000000000000000000 P e860dbdb17ce4d93882dcc337cd71bcf [sys.catalog]: SysCatalogTable state changed. Reason: New leader e860dbdb17ce4d93882dcc337cd71bcf. Latest consensus state: current_term: 1 leader_uuid: "e860dbdb17ce4d93882dcc337cd71bcf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e860dbdb17ce4d93882dcc337cd71bcf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33809 } } }
I20260807 08:51:46.793783 4707 sys_catalog.cc:455] T 00000000000000000000000000000000 P e860dbdb17ce4d93882dcc337cd71bcf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e860dbdb17ce4d93882dcc337cd71bcf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e860dbdb17ce4d93882dcc337cd71bcf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33809 } } }
I20260807 08:51:46.793857 4706 sys_catalog.cc:458] T 00000000000000000000000000000000 P e860dbdb17ce4d93882dcc337cd71bcf [sys.catalog]: This master's current role is: LEADER
I20260807 08:51:46.793869 4707 sys_catalog.cc:458] T 00000000000000000000000000000000 P e860dbdb17ce4d93882dcc337cd71bcf [sys.catalog]: This master's current role is: LEADER
I20260807 08:51:46.794131 4718 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:51:46.794667 4718 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:51:46.796418 4718 catalog_manager.cc:1347] Generated new cluster ID: 4cddd118759448c9a2ca563a9718da9b
I20260807 08:51:46.796464 4718 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:51:46.796499 4705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:46.790882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.181:34839 (local address 127.0.0.1:33809)
0807 08:51:46.791047 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:46.791056 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:46.791093 (+ 37us) server_negotiation.cc:407] Connection header received
0807 08:51:46.791869 (+ 776us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:46.791878 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:46.791952 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:46.792096 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:51:46.792299 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:46.793633 (+ 1334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:46.794736 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:46.795131 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:46.795224 (+ 93us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:46.795834 (+ 610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:51:46.795838 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:51:46.795838 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:51:46.796024 (+ 186us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:51:46.796060 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:51:46.796074 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:51:46.796147 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:46.796249 (+ 102us) server_negotiation.cc:299] Negotiation successful
0807 08:51:46.796275 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":72,"threads_started":1}
I20260807 08:51:46.797329 4734 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 08:51:46.801935 4705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:46.798703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.181:44701 (local address 127.0.0.1:33809)
0807 08:51:46.798732 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:46.798734 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:46.798941 (+ 207us) server_negotiation.cc:407] Connection header received
0807 08:51:46.798969 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:46.798970 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:46.798990 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:46.799056 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:51:46.799157 (+ 101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:46.799842 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:46.800748 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:46.801069 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:46.801153 (+ 84us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:46.801708 (+ 555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:51:46.801710 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:51:46.801710 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:51:46.801714 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:51:46.801728 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:51:46.801733 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:51:46.801802 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:46.801854 (+ 52us) server_negotiation.cc:299] Negotiation successful
0807 08:51:46.801867 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260807 08:51:46.816704 4718 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:51:46.817219 4718 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:51:46.822968 4718 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e860dbdb17ce4d93882dcc337cd71bcf: Generated new TSK 0
I20260807 08:51:46.823117 4718 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:51:46.825745 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 4600
W20260807 08:51:46.842608 4576 connection.cc:535] client connection to 127.0.0.1:33809 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33809): Connection reset by peer (error 104)
2026-08-07T08:51:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1392 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786092453486433-4734-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:51:46 dist-test-slave-6zbq krb5kdc[4749](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:51:46 dist-test-slave-6zbq krb5kdc[4749](info): set up 2 sockets
Aug 07 08:51:46 dist-test-slave-6zbq krb5kdc[4749](info): commencing operation
krb5kdc: starting...
W20260807 08:51:47.949311 4734 mini_kdc.cc:121] Time spent starting KDC: real 1.093s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786092453486433-4734-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786092453486433-4734-0/krb5kdc/test-user.keytab.
Aug 07 08:51:48 dist-test-slave-6zbq krb5kdc[4749](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092708, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:51:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:51:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786092453486433-4734-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786092453486433-4734-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 08:51:48.056217 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786092453486433-4734-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786092453486433-4734-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786092453486433-4734-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44099
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786092453486433-4734-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46119
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44099
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:51:48.134637 4772 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:51:48.134822 4772 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:51:48.134845 4772 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:51:48.135942 4772 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:51:48.135983 4772 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:51:48.135996 4772 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:51:48.136008 4772 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:51:48.136021 4772 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:51:48.137480 4772 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46119
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786092453486433-4734-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786092453486433-4734-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44099
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786092453486433-4734-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44099
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4772
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786092453486433-4734-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:51:48.137693 4772 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:51:48.137890 4772 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:51:48.142288 4781 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:51:48.142299 4784 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:51:48.142364 4772 server_base.cc:1034] running on GCE node
W20260807 08:51:48.142297 4780 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:51:48.142709 4772 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:51:48.142975 4772 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:51:48.144111 4772 hybrid_clock.cc:648] HybridClock initialized: now 1786092708144098 us; error 33 us; skew 500 ppm
Aug 07 08:51:48 dist-test-slave-6zbq krb5kdc[4749](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092708, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:51:48.146332 4772 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 08:51:48.146837 4772 webserver.cc:460] Webserver started at http://127.0.0.1:45989/ using document root <none> and password file <none>
I20260807 08:51:48.147080 4772 fs_manager.cc:359] Metadata directory not provided
I20260807 08:51:48.147145 4772 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:51:48.147293 4772 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:51:48.148375 4772 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786092453486433-4734-0/minicluster-data/master-0/data/instance:
uuid: "d8313ab7b8714feea58efb85b8f4b6e8"
format_stamp: "Formatted at 2026-08-07 08:51:48 on dist-test-slave-6zbq"
server_key: "d59fe28e1854e1f33dc9f77f72cd7147"
server_key_iv: "e0baf2afd80ca08794f10f973900c0c4"
server_key_version: "encryptionkey@0"
I20260807 08:51:48.148661 4772 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786092453486433-4734-0/minicluster-data/master-0/wal/instance:
uuid: "d8313ab7b8714feea58efb85b8f4b6e8"
format_stamp: "Formatted at 2026-08-07 08:51:48 on dist-test-slave-6zbq"
server_key: "d59fe28e1854e1f33dc9f77f72cd7147"
server_key_iv: "e0baf2afd80ca08794f10f973900c0c4"
server_key_version: "encryptionkey@0"
I20260807 08:51:48.149782 4772 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 08:51:48.150449 4795 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:51:48.150578 4772 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 08:51:48.150657 4772 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786092453486433-4734-0/minicluster-data/master-0/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786092453486433-4734-0/minicluster-data/master-0/wal
uuid: "d8313ab7b8714feea58efb85b8f4b6e8"
format_stamp: "Formatted at 2026-08-07 08:51:48 on dist-test-slave-6zbq"
server_key: "d59fe28e1854e1f33dc9f77f72cd7147"
server_key_iv: "e0baf2afd80ca08794f10f973900c0c4"
server_key_version: "encryptionkey@0"
I20260807 08:51:48.150723 4772 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786092453486433-4734-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786092453486433-4734-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786092453486433-4734-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:51:48.153764 4772 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:51:48.155402 4772 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:51:48.155517 4772 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:51:48.159277 4772 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44099
I20260807 08:51:48.159307 4864 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44099 every 8 connection(s)
I20260807 08:51:48.159581 4772 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
I20260807 08:51:48.160075 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 4772
I20260807 08:51:48.160210 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786092453486433-4734-0/minicluster-data/master-0/wal/instance
I20260807 08:51:48.160393 4734 external_mini_cluster.cc:1428] Setting key ffb5c8a4327ecbd917e3dd5558e75b6d
I20260807 08:51:48.160486 4865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:51:48.162612 4870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:48.160830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.181:35975 (local address 127.0.0.1:44099)
0807 08:51:48.160994 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:48.161002 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:48.161030 (+ 28us) server_negotiation.cc:407] Connection header received
0807 08:51:48.161450 (+ 420us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:48.161457 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:48.161482 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:48.161584 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:51:48.162168 (+ 584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:51:48.162171 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:51:48.162171 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:51:48.162175 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:51:48.162212 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:51:48.162222 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:51:48.162274 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:48.162398 (+ 124us) server_negotiation.cc:299] Negotiation successful
0807 08:51:48.162444 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":83,"threads_started":1}
I20260807 08:51:48.163530 4734 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 08:51:48.163563 4865 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8313ab7b8714feea58efb85b8f4b6e8: Bootstrap starting.
I20260807 08:51:48.164213 4865 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d8313ab7b8714feea58efb85b8f4b6e8: Neither blocks nor log segments found. Creating new log.
I20260807 08:51:48.164542 4865 log.cc:824] T 00000000000000000000000000000000 P d8313ab7b8714feea58efb85b8f4b6e8: Log is configured to *not* fsync() on all Append() calls
I20260807 08:51:48.165249 4865 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8313ab7b8714feea58efb85b8f4b6e8: No bootstrap required, opened a new log
I20260807 08:51:48.165926 4870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:48.165005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.181:60175 (local address 127.0.0.1:44099)
0807 08:51:48.165034 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:48.165035 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:48.165143 (+ 108us) server_negotiation.cc:407] Connection header received
0807 08:51:48.165199 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:48.165199 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:48.165203 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:48.165247 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:51:48.165730 (+ 483us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:51:48.165731 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:51:48.165732 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:51:48.165735 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:51:48.165747 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:51:48.165750 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:51:48.165803 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:48.165858 (+ 55us) server_negotiation.cc:299] Negotiation successful
0807 08:51:48.165870 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12}
I20260807 08:51:48.167227 4865 raft_consensus.cc:348] T 00000000000000000000000000000000 P d8313ab7b8714feea58efb85b8f4b6e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8313ab7b8714feea58efb85b8f4b6e8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44099 } }
I20260807 08:51:48.167353 4865 raft_consensus.cc:374] T 00000000000000000000000000000000 P d8313ab7b8714feea58efb85b8f4b6e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:51:48.167376 4865 raft_consensus.cc:729] T 00000000000000000000000000000000 P d8313ab7b8714feea58efb85b8f4b6e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8313ab7b8714feea58efb85b8f4b6e8, State: Initialized, Role: FOLLOWER
I20260807 08:51:48.167456 4865 consensus_queue.cc:260] T 00000000000000000000000000000000 P d8313ab7b8714feea58efb85b8f4b6e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8313ab7b8714feea58efb85b8f4b6e8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44099 } }
I20260807 08:51:48.167513 4865 raft_consensus.cc:388] T 00000000000000000000000000000000 P d8313ab7b8714feea58efb85b8f4b6e8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:51:48.167538 4865 raft_consensus.cc:482] T 00000000000000000000000000000000 P d8313ab7b8714feea58efb85b8f4b6e8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:51:48.167567 4865 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d8313ab7b8714feea58efb85b8f4b6e8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:51:48.168023 4865 raft_consensus.cc:504] T 00000000000000000000000000000000 P d8313ab7b8714feea58efb85b8f4b6e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8313ab7b8714feea58efb85b8f4b6e8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44099 } }
I20260807 08:51:48.168107 4865 leader_election.cc:302] T 00000000000000000000000000000000 P d8313ab7b8714feea58efb85b8f4b6e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d8313ab7b8714feea58efb85b8f4b6e8; no voters:
I20260807 08:51:48.168223 4865 leader_election.cc:288] T 00000000000000000000000000000000 P d8313ab7b8714feea58efb85b8f4b6e8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:51:48.168286 4888 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d8313ab7b8714feea58efb85b8f4b6e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:51:48.168423 4865 sys_catalog.cc:564] T 00000000000000000000000000000000 P d8313ab7b8714feea58efb85b8f4b6e8 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:51:48.168434 4888 raft_consensus.cc:686] T 00000000000000000000000000000000 P d8313ab7b8714feea58efb85b8f4b6e8 [term 1 LEADER]: Becoming Leader. State: Replica: d8313ab7b8714feea58efb85b8f4b6e8, State: Running, Role: LEADER
I20260807 08:51:48.168535 4888 consensus_queue.cc:237] T 00000000000000000000000000000000 P d8313ab7b8714feea58efb85b8f4b6e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8313ab7b8714feea58efb85b8f4b6e8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44099 } }
I20260807 08:51:48.168830 4888 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8313ab7b8714feea58efb85b8f4b6e8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d8313ab7b8714feea58efb85b8f4b6e8. Latest consensus state: current_term: 1 leader_uuid: "d8313ab7b8714feea58efb85b8f4b6e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8313ab7b8714feea58efb85b8f4b6e8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44099 } } }
I20260807 08:51:48.168867 4893 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8313ab7b8714feea58efb85b8f4b6e8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d8313ab7b8714feea58efb85b8f4b6e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8313ab7b8714feea58efb85b8f4b6e8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44099 } } }
I20260807 08:51:48.168968 4888 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8313ab7b8714feea58efb85b8f4b6e8 [sys.catalog]: This master's current role is: LEADER
I20260807 08:51:48.169179 4893 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8313ab7b8714feea58efb85b8f4b6e8 [sys.catalog]: This master's current role is: LEADER
I20260807 08:51:48.169195 4901 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:51:48.169938 4901 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:51:48.171259 4901 catalog_manager.cc:1347] Generated new cluster ID: 5e8c4705d2f74912bba7e874837db68e
I20260807 08:51:48.171311 4901 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:51:48.193027 4901 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:51:48.193163 4901 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:51:48.195531 4901 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d8313ab7b8714feea58efb85b8f4b6e8: Generated new TSK 0
I20260807 08:51:48.195686 4901 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:51:48.204332 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 4772
2026-08-07T08:51:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1372 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786092453486433-4734-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:51:48 dist-test-slave-6zbq krb5kdc[4919](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:51:48 dist-test-slave-6zbq krb5kdc[4919](info): set up 2 sockets
Aug 07 08:51:48 dist-test-slave-6zbq krb5kdc[4919](info): commencing operation
krb5kdc: starting...
W20260807 08:51:49.335148 4734 mini_kdc.cc:121] Time spent starting KDC: real 1.107s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786092453486433-4734-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786092453486433-4734-0/krb5kdc/test-user.keytab.
Aug 07 08:51:49 dist-test-slave-6zbq krb5kdc[4919](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092709, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:51:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:51:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.159.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.159.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.159.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab.
Entry for principal kudu/127.4.159.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab.
I20260807 08:51:49.440765 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786092453486433-4734-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786092453486433-4734-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786092453486433-4734-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.159.190:38381
--webserver_interface=127.4.159.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab
--principal=kudu/127.4.159.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:35907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.159.190:38381
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:51:49.518887 4936 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:51:49.519076 4936 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:51:49.519107 4936 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:51:49.520200 4936 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:51:49.520237 4936 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:51:49.520256 4936 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:51:49.520273 4936 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:51:49.520289 4936 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:51:49.521727 4936 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:35907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786092453486433-4734-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786092453486433-4734-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.159.190:38381
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab
--principal=kudu/127.4.159.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.159.190:38381
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.159.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4936
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786092453486433-4734-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:51:49.521955 4936 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:51:49.522152 4936 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:51:49.526502 4946 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:51:49.526548 4945 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:51:49.526525 4950 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:51:49.526584 4936 server_base.cc:1034] running on GCE node
I20260807 08:51:49.526892 4936 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:51:49.527159 4936 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:51:49.528291 4936 hybrid_clock.cc:648] HybridClock initialized: now 1786092709528279 us; error 31 us; skew 500 ppm
Aug 07 08:51:49 dist-test-slave-6zbq krb5kdc[4919](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092709, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:51:49.530580 4936 init.cc:377] Logged in from keytab as kudu/127.4.159.190@KRBTEST.COM (short username kudu)
I20260807 08:51:49.530985 4936 webserver.cc:460] Webserver started at http://127.4.159.190:33891/ using document root <none> and password file <none>
I20260807 08:51:49.531193 4936 fs_manager.cc:359] Metadata directory not provided
I20260807 08:51:49.531239 4936 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:51:49.531358 4936 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:51:49.532294 4936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786092453486433-4734-0/minicluster-data/master-0/data/instance:
uuid: "ed1f8f03d49f4085a903ad885a5fd0ba"
format_stamp: "Formatted at 2026-08-07 08:51:49 on dist-test-slave-6zbq"
server_key: "c0668788cdd857d258aedfbcf505f076"
server_key_iv: "5d111381c8b5aa99390e440006974a14"
server_key_version: "encryptionkey@0"
I20260807 08:51:49.532603 4936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786092453486433-4734-0/minicluster-data/master-0/wal/instance:
uuid: "ed1f8f03d49f4085a903ad885a5fd0ba"
format_stamp: "Formatted at 2026-08-07 08:51:49 on dist-test-slave-6zbq"
server_key: "c0668788cdd857d258aedfbcf505f076"
server_key_iv: "5d111381c8b5aa99390e440006974a14"
server_key_version: "encryptionkey@0"
I20260807 08:51:49.533769 4936 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 08:51:49.534426 4961 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:51:49.534564 4936 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 08:51:49.534651 4936 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786092453486433-4734-0/minicluster-data/master-0/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786092453486433-4734-0/minicluster-data/master-0/wal
uuid: "ed1f8f03d49f4085a903ad885a5fd0ba"
format_stamp: "Formatted at 2026-08-07 08:51:49 on dist-test-slave-6zbq"
server_key: "c0668788cdd857d258aedfbcf505f076"
server_key_iv: "5d111381c8b5aa99390e440006974a14"
server_key_version: "encryptionkey@0"
I20260807 08:51:49.534716 4936 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786092453486433-4734-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786092453486433-4734-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786092453486433-4734-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:51:49.552336 4936 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:51:49.553856 4936 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:51:49.553975 4936 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:51:49.557716 4936 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.190:38381
I20260807 08:51:49.557751 5034 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.190:38381 every 8 connection(s)
I20260807 08:51:49.558046 4936 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
I20260807 08:51:49.558959 5035 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:51:49.561064 5035 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed1f8f03d49f4085a903ad885a5fd0ba: Bootstrap starting.
I20260807 08:51:49.561586 5035 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ed1f8f03d49f4085a903ad885a5fd0ba: Neither blocks nor log segments found. Creating new log.
I20260807 08:51:49.561847 5035 log.cc:824] T 00000000000000000000000000000000 P ed1f8f03d49f4085a903ad885a5fd0ba: Log is configured to *not* fsync() on all Append() calls
I20260807 08:51:49.562558 5035 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed1f8f03d49f4085a903ad885a5fd0ba: No bootstrap required, opened a new log
I20260807 08:51:49.564087 5035 raft_consensus.cc:348] T 00000000000000000000000000000000 P ed1f8f03d49f4085a903ad885a5fd0ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed1f8f03d49f4085a903ad885a5fd0ba" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 38381 } }
I20260807 08:51:49.564221 5035 raft_consensus.cc:374] T 00000000000000000000000000000000 P ed1f8f03d49f4085a903ad885a5fd0ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:51:49.564242 5035 raft_consensus.cc:729] T 00000000000000000000000000000000 P ed1f8f03d49f4085a903ad885a5fd0ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed1f8f03d49f4085a903ad885a5fd0ba, State: Initialized, Role: FOLLOWER
I20260807 08:51:49.564342 5035 consensus_queue.cc:260] T 00000000000000000000000000000000 P ed1f8f03d49f4085a903ad885a5fd0ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed1f8f03d49f4085a903ad885a5fd0ba" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 38381 } }
I20260807 08:51:49.564391 5035 raft_consensus.cc:388] T 00000000000000000000000000000000 P ed1f8f03d49f4085a903ad885a5fd0ba [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:51:49.564410 5035 raft_consensus.cc:482] T 00000000000000000000000000000000 P ed1f8f03d49f4085a903ad885a5fd0ba [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:51:49.564435 5035 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ed1f8f03d49f4085a903ad885a5fd0ba [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:51:49.564531 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 4936
I20260807 08:51:49.564620 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786092453486433-4734-0/minicluster-data/master-0/wal/instance
I20260807 08:51:49.564736 4734 external_mini_cluster.cc:1428] Setting key ea4cada2e7f27df87284f596df2fda5c
I20260807 08:51:49.565326 5035 raft_consensus.cc:504] T 00000000000000000000000000000000 P ed1f8f03d49f4085a903ad885a5fd0ba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed1f8f03d49f4085a903ad885a5fd0ba" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 38381 } }
I20260807 08:51:49.565438 5035 leader_election.cc:302] T 00000000000000000000000000000000 P ed1f8f03d49f4085a903ad885a5fd0ba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ed1f8f03d49f4085a903ad885a5fd0ba; no voters:
I20260807 08:51:49.565593 5035 leader_election.cc:288] T 00000000000000000000000000000000 P ed1f8f03d49f4085a903ad885a5fd0ba [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:51:49.565629 5042 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ed1f8f03d49f4085a903ad885a5fd0ba [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:51:49.565802 5042 raft_consensus.cc:686] T 00000000000000000000000000000000 P ed1f8f03d49f4085a903ad885a5fd0ba [term 1 LEADER]: Becoming Leader. State: Replica: ed1f8f03d49f4085a903ad885a5fd0ba, State: Running, Role: LEADER
I20260807 08:51:49.565917 5035 sys_catalog.cc:564] T 00000000000000000000000000000000 P ed1f8f03d49f4085a903ad885a5fd0ba [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:51:49.565932 5042 consensus_queue.cc:237] T 00000000000000000000000000000000 P ed1f8f03d49f4085a903ad885a5fd0ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed1f8f03d49f4085a903ad885a5fd0ba" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 38381 } }
I20260807 08:51:49.566267 5042 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed1f8f03d49f4085a903ad885a5fd0ba [sys.catalog]: SysCatalogTable state changed. Reason: New leader ed1f8f03d49f4085a903ad885a5fd0ba. Latest consensus state: current_term: 1 leader_uuid: "ed1f8f03d49f4085a903ad885a5fd0ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed1f8f03d49f4085a903ad885a5fd0ba" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 38381 } } }
I20260807 08:51:49.566394 5042 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed1f8f03d49f4085a903ad885a5fd0ba [sys.catalog]: This master's current role is: LEADER
I20260807 08:51:49.566655 5054 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:51:49.567210 5054 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:51:49.566274 5043 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed1f8f03d49f4085a903ad885a5fd0ba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ed1f8f03d49f4085a903ad885a5fd0ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed1f8f03d49f4085a903ad885a5fd0ba" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 38381 } } }
I20260807 08:51:49.567574 5043 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed1f8f03d49f4085a903ad885a5fd0ba [sys.catalog]: This master's current role is: LEADER
I20260807 08:51:49.568681 5054 catalog_manager.cc:1347] Generated new cluster ID: 73aef32261324b54b0ef1b3173dae6ac
I20260807 08:51:49.568733 5054 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 08:51:49 dist-test-slave-6zbq krb5kdc[4919](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092709, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.159.190@KRBTEST.COM
I20260807 08:51:49.575651 5040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:49.565172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60171 (local address 127.4.159.190:38381)
0807 08:51:49.565323 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:49.565332 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:49.565355 (+ 23us) server_negotiation.cc:407] Connection header received
0807 08:51:49.565459 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:49.565465 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:49.565519 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:49.565653 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:51:49.566361 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:49.567723 (+ 1362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:49.568681 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:49.569228 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:49.569332 (+ 104us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:49.572058 (+ 2726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:51:49.572063 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:51:49.572063 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:51:49.572075 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:51:49.573993 (+ 1918us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:51:49.574414 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:51:49.574417 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:51:49.574418 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:51:49.574537 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:51:49.574747 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:51:49.574748 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:51:49.574749 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:51:49.575062 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:51:49.575229 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:49.575340 (+ 111us) server_negotiation.cc:299] Negotiation successful
0807 08:51:49.575443 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":67,"threads_started":1}
I20260807 08:51:49.576043 5054 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:51:49.576593 5054 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:51:49.576874 4734 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 08:51:49.583422 5054 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ed1f8f03d49f4085a903ad885a5fd0ba: Generated new TSK 0
I20260807 08:51:49.583572 5054 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:51:49.584645 5040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:49.578175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44965 (local address 127.4.159.190:38381)
0807 08:51:49.578207 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:49.578209 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:49.578307 (+ 98us) server_negotiation.cc:407] Connection header received
0807 08:51:49.578369 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:49.578370 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:49.578379 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:49.578443 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:51:49.578967 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:49.579727 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:49.580670 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:49.581012 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:49.581073 (+ 61us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:49.582256 (+ 1183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:51:49.582257 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:51:49.582257 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:51:49.582265 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:51:49.583679 (+ 1414us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:51:49.583982 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:51:49.583983 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:51:49.583983 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:51:49.584038 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:51:49.584214 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:51:49.584215 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:51:49.584215 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:51:49.584331 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:51:49.584425 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:49.584503 (+ 78us) server_negotiation.cc:299] Negotiation successful
0807 08:51:49.584560 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":15}
I20260807 08:51:49.586397 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 4936
W20260807 08:51:49.599946 4916 connection.cc:535] client connection to 127.4.159.190:38381 recv error: Network error: failed to read from TLS socket (remote: 127.4.159.190:38381): Connection reset by peer (error 104)
2026-08-07T08:51:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1385 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786092453486433-4734-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:51:49 dist-test-slave-6zbq krb5kdc[5082](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:51:49 dist-test-slave-6zbq krb5kdc[5082](info): set up 2 sockets
Aug 07 08:51:49 dist-test-slave-6zbq krb5kdc[5082](info): commencing operation
krb5kdc: starting...
W20260807 08:51:50.707993 4734 mini_kdc.cc:121] Time spent starting KDC: real 1.095s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786092453486433-4734-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786092453486433-4734-0/krb5kdc/test-user.keytab.
Aug 07 08:51:50 dist-test-slave-6zbq krb5kdc[5082](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092710, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:51:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:51:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.159.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.159.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.159.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab.
Entry for principal kudu/127.4.159.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab.
I20260807 08:51:50.812220 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786092453486433-4734-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786092453486433-4734-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786092453486433-4734-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.159.190:37997
--webserver_interface=127.4.159.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab
--principal=kudu/127.4.159.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:39089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.159.190:37997
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:51:50.890183 5100 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:51:50.890408 5100 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:51:50.890439 5100 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:51:50.891573 5100 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:51:50.891613 5100 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:51:50.891626 5100 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:51:50.891639 5100 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:51:50.891664 5100 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:51:50.893095 5100 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:39089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786092453486433-4734-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786092453486433-4734-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.159.190:37997
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab
--principal=kudu/127.4.159.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.159.190:37997
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.159.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5100
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786092453486433-4734-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:51:50.893316 5100 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:51:50.893507 5100 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:51:50.897853 5108 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:51:50.897876 5109 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:51:50.897862 5112 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:51:50.898084 5100 server_base.cc:1034] running on GCE node
I20260807 08:51:50.898289 5100 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:51:50.898566 5100 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:51:50.899691 5100 hybrid_clock.cc:648] HybridClock initialized: now 1786092710899684 us; error 26 us; skew 500 ppm
Aug 07 08:51:50 dist-test-slave-6zbq krb5kdc[5082](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092710, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:51:50.905628 5100 init.cc:377] Logged in from keytab as kudu/127.4.159.190@KRBTEST.COM (short username kudu)
I20260807 08:51:50.906066 5100 webserver.cc:460] Webserver started at http://127.4.159.190:42657/ using document root <none> and password file <none>
I20260807 08:51:50.906268 5100 fs_manager.cc:359] Metadata directory not provided
I20260807 08:51:50.906317 5100 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:51:50.906462 5100 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:51:50.907411 5100 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786092453486433-4734-0/minicluster-data/master-0/data/instance:
uuid: "8a8fade5307f4b1fa26657d87a054337"
format_stamp: "Formatted at 2026-08-07 08:51:50 on dist-test-slave-6zbq"
server_key: "4576b361dba79e1ee3e54fc8cf749374"
server_key_iv: "17dc4b71ac0af2b10422d7856f3c2c6c"
server_key_version: "encryptionkey@0"
I20260807 08:51:50.907723 5100 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786092453486433-4734-0/minicluster-data/master-0/wal/instance:
uuid: "8a8fade5307f4b1fa26657d87a054337"
format_stamp: "Formatted at 2026-08-07 08:51:50 on dist-test-slave-6zbq"
server_key: "4576b361dba79e1ee3e54fc8cf749374"
server_key_iv: "17dc4b71ac0af2b10422d7856f3c2c6c"
server_key_version: "encryptionkey@0"
I20260807 08:51:50.908883 5100 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 08:51:50.909502 5120 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:51:50.909642 5100 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 08:51:50.909724 5100 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786092453486433-4734-0/minicluster-data/master-0/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786092453486433-4734-0/minicluster-data/master-0/wal
uuid: "8a8fade5307f4b1fa26657d87a054337"
format_stamp: "Formatted at 2026-08-07 08:51:50 on dist-test-slave-6zbq"
server_key: "4576b361dba79e1ee3e54fc8cf749374"
server_key_iv: "17dc4b71ac0af2b10422d7856f3c2c6c"
server_key_version: "encryptionkey@0"
I20260807 08:51:50.909791 5100 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786092453486433-4734-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786092453486433-4734-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786092453486433-4734-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:51:50.929337 5100 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:51:50.931241 5100 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:51:50.931367 5100 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:51:50.934934 5100 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.190:37997
I20260807 08:51:50.934966 5199 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.190:37997 every 8 connection(s)
I20260807 08:51:50.935245 5100 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
I20260807 08:51:50.936028 5201 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:51:50.936178 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 5100
I20260807 08:51:50.936292 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786092453486433-4734-0/minicluster-data/master-0/wal/instance
I20260807 08:51:50.936501 4734 external_mini_cluster.cc:1428] Setting key 6f5c994bf18db434c9cf65e2e55eb95e
I20260807 08:51:50.938532 5201 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a8fade5307f4b1fa26657d87a054337: Bootstrap starting.
I20260807 08:51:50.939129 5201 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8a8fade5307f4b1fa26657d87a054337: Neither blocks nor log segments found. Creating new log.
I20260807 08:51:50.939437 5201 log.cc:824] T 00000000000000000000000000000000 P 8a8fade5307f4b1fa26657d87a054337: Log is configured to *not* fsync() on all Append() calls
I20260807 08:51:50.940079 5201 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a8fade5307f4b1fa26657d87a054337: No bootstrap required, opened a new log
I20260807 08:51:50.941497 5209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:50.936971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55851 (local address 127.4.159.190:37997)
0807 08:51:50.937120 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:50.937127 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:50.937149 (+ 22us) server_negotiation.cc:407] Connection header received
0807 08:51:50.937534 (+ 385us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:50.937540 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:50.937581 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:50.937687 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:51:50.937911 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:50.938852 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:50.939831 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:50.940234 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:50.940342 (+ 108us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:50.940942 (+ 600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:51:50.940946 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:51:50.940946 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:51:50.941100 (+ 154us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:51:50.941141 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:51:50.941151 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:51:50.941218 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:50.941323 (+ 105us) server_negotiation.cc:299] Negotiation successful
0807 08:51:50.941348 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":51,"threads_started":1}
I20260807 08:51:50.942049 5201 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8a8fade5307f4b1fa26657d87a054337 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a8fade5307f4b1fa26657d87a054337" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 37997 } }
I20260807 08:51:50.942193 5201 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8a8fade5307f4b1fa26657d87a054337 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:51:50.942235 5201 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8a8fade5307f4b1fa26657d87a054337 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a8fade5307f4b1fa26657d87a054337, State: Initialized, Role: FOLLOWER
I20260807 08:51:50.942380 5201 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8a8fade5307f4b1fa26657d87a054337 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a8fade5307f4b1fa26657d87a054337" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 37997 } }
I20260807 08:51:50.942464 5201 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8a8fade5307f4b1fa26657d87a054337 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:51:50.942459 4734 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 08:51:50.942503 5201 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8a8fade5307f4b1fa26657d87a054337 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:51:50.942548 5201 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8a8fade5307f4b1fa26657d87a054337 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:51:50.943164 5201 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8a8fade5307f4b1fa26657d87a054337 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a8fade5307f4b1fa26657d87a054337" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 37997 } }
I20260807 08:51:50.943284 5201 leader_election.cc:302] T 00000000000000000000000000000000 P 8a8fade5307f4b1fa26657d87a054337 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8a8fade5307f4b1fa26657d87a054337; no voters:
I20260807 08:51:50.943449 5201 leader_election.cc:288] T 00000000000000000000000000000000 P 8a8fade5307f4b1fa26657d87a054337 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:51:50.943521 5211 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8a8fade5307f4b1fa26657d87a054337 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:51:50.943682 5211 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8a8fade5307f4b1fa26657d87a054337 [term 1 LEADER]: Becoming Leader. State: Replica: 8a8fade5307f4b1fa26657d87a054337, State: Running, Role: LEADER
I20260807 08:51:50.943735 5201 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8a8fade5307f4b1fa26657d87a054337 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:51:50.943809 5211 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8a8fade5307f4b1fa26657d87a054337 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a8fade5307f4b1fa26657d87a054337" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 37997 } }
I20260807 08:51:50.944217 5211 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a8fade5307f4b1fa26657d87a054337 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8a8fade5307f4b1fa26657d87a054337. Latest consensus state: current_term: 1 leader_uuid: "8a8fade5307f4b1fa26657d87a054337" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a8fade5307f4b1fa26657d87a054337" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 37997 } } }
I20260807 08:51:50.944307 5211 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a8fade5307f4b1fa26657d87a054337 [sys.catalog]: This master's current role is: LEADER
I20260807 08:51:50.944555 5229 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:51:50.945096 5229 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:51:50.945276 5223 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a8fade5307f4b1fa26657d87a054337 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8a8fade5307f4b1fa26657d87a054337" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a8fade5307f4b1fa26657d87a054337" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 37997 } } }
I20260807 08:51:50.945505 5223 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a8fade5307f4b1fa26657d87a054337 [sys.catalog]: This master's current role is: LEADER
I20260807 08:51:50.946926 5229 catalog_manager.cc:1347] Generated new cluster ID: e8258a142e9c47f08a07bd47efcc54fb
I20260807 08:51:50.946978 5229 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:51:50.947204 5209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:50.943906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50491 (local address 127.4.159.190:37997)
0807 08:51:50.943936 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:50.943938 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:50.944070 (+ 132us) server_negotiation.cc:407] Connection header received
0807 08:51:50.944147 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:50.944148 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:50.944157 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:50.944208 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:51:50.944411 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:50.945172 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:50.945973 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:50.946237 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:50.946310 (+ 73us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:50.946921 (+ 611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:51:50.946923 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:51:50.946923 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:51:50.946927 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:51:50.946940 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:51:50.946944 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:51:50.947003 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:50.947128 (+ 125us) server_negotiation.cc:299] Negotiation successful
0807 08:51:50.947139 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260807 08:51:50.962750 5229 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:51:50.963204 5229 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:51:50.964978 5229 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8a8fade5307f4b1fa26657d87a054337: Generated new TSK 0
I20260807 08:51:50.965119 5229 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:51:50.967764 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 5100
W20260807 08:51:50.980792 5079 connection.cc:535] client connection to 127.4.159.190:37997 recv error: Network error: failed to read from TLS socket (remote: 127.4.159.190:37997): Connection reset by peer (error 104)
2026-08-07T08:51:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1380 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786092453486433-4734-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:51:51 dist-test-slave-6zbq krb5kdc[5250](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:51:51 dist-test-slave-6zbq krb5kdc[5250](info): set up 2 sockets
Aug 07 08:51:51 dist-test-slave-6zbq krb5kdc[5250](info): commencing operation
krb5kdc: starting...
W20260807 08:51:52.088481 4734 mini_kdc.cc:121] Time spent starting KDC: real 1.095s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786092453486433-4734-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786092453486433-4734-0/krb5kdc/test-user.keytab.
Aug 07 08:51:52 dist-test-slave-6zbq krb5kdc[5250](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092712, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:51:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:51:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.159.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.159.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.159.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab.
Entry for principal kudu/127.4.159.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab.
I20260807 08:51:52.194381 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786092453486433-4734-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786092453486433-4734-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786092453486433-4734-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.159.190:41035
--webserver_interface=127.4.159.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab
--principal=kudu/127.4.159.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:41159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.159.190:41035
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:51:52.273445 5274 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:51:52.273620 5274 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:51:52.273640 5274 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:51:52.274709 5274 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:51:52.274742 5274 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:51:52.274755 5274 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:51:52.274765 5274 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:51:52.274780 5274 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:51:52.276189 5274 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:41159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786092453486433-4734-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786092453486433-4734-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.159.190:41035
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab
--principal=kudu/127.4.159.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.159.190:41035
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.159.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5274
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786092453486433-4734-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:51:52.276364 5274 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:51:52.276530 5274 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:51:52.280870 5291 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:51:52.280886 5284 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:51:52.280903 5274 server_base.cc:1034] running on GCE node
W20260807 08:51:52.280886 5286 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:51:52.281306 5274 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:51:52.281580 5274 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:51:52.282720 5274 hybrid_clock.cc:648] HybridClock initialized: now 1786092712282707 us; error 29 us; skew 500 ppm
Aug 07 08:51:52 dist-test-slave-6zbq krb5kdc[5250](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092712, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:51:52.285113 5274 init.cc:377] Logged in from keytab as kudu/127.4.159.190@KRBTEST.COM (short username kudu)
I20260807 08:51:52.285503 5274 webserver.cc:460] Webserver started at http://127.4.159.190:38013/ using document root <none> and password file <none>
I20260807 08:51:52.285707 5274 fs_manager.cc:359] Metadata directory not provided
I20260807 08:51:52.285761 5274 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:51:52.285885 5274 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:51:52.286909 5274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786092453486433-4734-0/minicluster-data/master-0/data/instance:
uuid: "d523076f2aa04b67904e5d8519bf1dea"
format_stamp: "Formatted at 2026-08-07 08:51:52 on dist-test-slave-6zbq"
server_key: "03ae9e0a3df7da3446c40d976c3a86b5"
server_key_iv: "25b4df75e1b20b171649ffb0624df124"
server_key_version: "encryptionkey@0"
I20260807 08:51:52.287222 5274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786092453486433-4734-0/minicluster-data/master-0/wal/instance:
uuid: "d523076f2aa04b67904e5d8519bf1dea"
format_stamp: "Formatted at 2026-08-07 08:51:52 on dist-test-slave-6zbq"
server_key: "03ae9e0a3df7da3446c40d976c3a86b5"
server_key_iv: "25b4df75e1b20b171649ffb0624df124"
server_key_version: "encryptionkey@0"
I20260807 08:51:52.288542 5274 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 08:51:52.289197 5299 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:51:52.289366 5274 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 08:51:52.289451 5274 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786092453486433-4734-0/minicluster-data/master-0/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786092453486433-4734-0/minicluster-data/master-0/wal
uuid: "d523076f2aa04b67904e5d8519bf1dea"
format_stamp: "Formatted at 2026-08-07 08:51:52 on dist-test-slave-6zbq"
server_key: "03ae9e0a3df7da3446c40d976c3a86b5"
server_key_iv: "25b4df75e1b20b171649ffb0624df124"
server_key_version: "encryptionkey@0"
I20260807 08:51:52.289511 5274 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786092453486433-4734-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786092453486433-4734-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786092453486433-4734-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:51:52.292672 5274 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:51:52.294909 5274 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:51:52.295013 5274 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:51:52.298694 5274 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.190:41035
I20260807 08:51:52.298733 5387 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.190:41035 every 8 connection(s)
I20260807 08:51:52.299007 5274 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
I20260807 08:51:52.299906 5388 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:51:52.302029 5388 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d523076f2aa04b67904e5d8519bf1dea: Bootstrap starting.
I20260807 08:51:52.302590 5388 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d523076f2aa04b67904e5d8519bf1dea: Neither blocks nor log segments found. Creating new log.
I20260807 08:51:52.302855 5388 log.cc:824] T 00000000000000000000000000000000 P d523076f2aa04b67904e5d8519bf1dea: Log is configured to *not* fsync() on all Append() calls
I20260807 08:51:52.303450 5388 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d523076f2aa04b67904e5d8519bf1dea: No bootstrap required, opened a new log
I20260807 08:51:52.305034 5388 raft_consensus.cc:348] T 00000000000000000000000000000000 P d523076f2aa04b67904e5d8519bf1dea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d523076f2aa04b67904e5d8519bf1dea" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 41035 } }
I20260807 08:51:52.305172 5388 raft_consensus.cc:374] T 00000000000000000000000000000000 P d523076f2aa04b67904e5d8519bf1dea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:51:52.305195 5388 raft_consensus.cc:729] T 00000000000000000000000000000000 P d523076f2aa04b67904e5d8519bf1dea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d523076f2aa04b67904e5d8519bf1dea, State: Initialized, Role: FOLLOWER
I20260807 08:51:52.305330 5388 consensus_queue.cc:260] T 00000000000000000000000000000000 P d523076f2aa04b67904e5d8519bf1dea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d523076f2aa04b67904e5d8519bf1dea" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 41035 } }
I20260807 08:51:52.305403 5388 raft_consensus.cc:388] T 00000000000000000000000000000000 P d523076f2aa04b67904e5d8519bf1dea [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:51:52.305438 5388 raft_consensus.cc:482] T 00000000000000000000000000000000 P d523076f2aa04b67904e5d8519bf1dea [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:51:52.305485 5388 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d523076f2aa04b67904e5d8519bf1dea [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:51:52.306258 5388 raft_consensus.cc:504] T 00000000000000000000000000000000 P d523076f2aa04b67904e5d8519bf1dea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d523076f2aa04b67904e5d8519bf1dea" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 41035 } }
I20260807 08:51:52.306378 5388 leader_election.cc:302] T 00000000000000000000000000000000 P d523076f2aa04b67904e5d8519bf1dea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d523076f2aa04b67904e5d8519bf1dea; no voters:
I20260807 08:51:52.306528 5388 leader_election.cc:288] T 00000000000000000000000000000000 P d523076f2aa04b67904e5d8519bf1dea [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:51:52.306571 5395 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d523076f2aa04b67904e5d8519bf1dea [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:51:52.306725 5395 raft_consensus.cc:686] T 00000000000000000000000000000000 P d523076f2aa04b67904e5d8519bf1dea [term 1 LEADER]: Becoming Leader. State: Replica: d523076f2aa04b67904e5d8519bf1dea, State: Running, Role: LEADER
I20260807 08:51:52.306787 5388 sys_catalog.cc:564] T 00000000000000000000000000000000 P d523076f2aa04b67904e5d8519bf1dea [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:51:52.306855 5395 consensus_queue.cc:237] T 00000000000000000000000000000000 P d523076f2aa04b67904e5d8519bf1dea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d523076f2aa04b67904e5d8519bf1dea" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 41035 } }
I20260807 08:51:52.307193 5398 sys_catalog.cc:455] T 00000000000000000000000000000000 P d523076f2aa04b67904e5d8519bf1dea [sys.catalog]: SysCatalogTable state changed. Reason: New leader d523076f2aa04b67904e5d8519bf1dea. Latest consensus state: current_term: 1 leader_uuid: "d523076f2aa04b67904e5d8519bf1dea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d523076f2aa04b67904e5d8519bf1dea" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 41035 } } }
I20260807 08:51:52.307288 5398 sys_catalog.cc:458] T 00000000000000000000000000000000 P d523076f2aa04b67904e5d8519bf1dea [sys.catalog]: This master's current role is: LEADER
I20260807 08:51:52.307185 5396 sys_catalog.cc:455] T 00000000000000000000000000000000 P d523076f2aa04b67904e5d8519bf1dea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d523076f2aa04b67904e5d8519bf1dea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d523076f2aa04b67904e5d8519bf1dea" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 41035 } } }
I20260807 08:51:52.307411 5396 sys_catalog.cc:458] T 00000000000000000000000000000000 P d523076f2aa04b67904e5d8519bf1dea [sys.catalog]: This master's current role is: LEADER
I20260807 08:51:52.307802 5411 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:51:52.308153 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 5274
I20260807 08:51:52.308281 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786092453486433-4734-0/minicluster-data/master-0/wal/instance
I20260807 08:51:52.308271 5411 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:51:52.308466 4734 external_mini_cluster.cc:1428] Setting key 2984b42017ddf01e6cee27bd4610ac9f
I20260807 08:51:52.309877 5411 catalog_manager.cc:1347] Generated new cluster ID: 866140ab82ba4d8e8bb02706416b9d67
I20260807 08:51:52.309921 5411 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:51:52.310835 5427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:52.308891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50051 (local address 127.4.159.190:41035)
0807 08:51:52.309542 (+ 651us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:52.309550 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:52.309573 (+ 23us) server_negotiation.cc:407] Connection header received
0807 08:51:52.309660 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:52.309665 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:52.309686 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:52.309793 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:51:52.310442 (+ 649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:51:52.310446 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:51:52.310446 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:51:52.310451 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:51:52.310489 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:51:52.310500 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:51:52.310552 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:52.310624 (+ 72us) server_negotiation.cc:299] Negotiation successful
0807 08:51:52.310665 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":595,"thread_start_us":61,"threads_started":1}
I20260807 08:51:52.315375 4734 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 08:51:52.317294 5411 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:51:52.317431 5411 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:51:52.319308 5427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:52.316627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47355 (local address 127.4.159.190:41035)
0807 08:51:52.316659 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:52.316661 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:52.318513 (+ 1852us) server_negotiation.cc:407] Connection header received
0807 08:51:52.318569 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:52.318570 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:52.318576 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:52.318615 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:51:52.319120 (+ 505us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:51:52.319122 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:51:52.319122 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:51:52.319126 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:51:52.319139 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:51:52.319143 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:51:52.319188 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:52.319227 (+ 39us) server_negotiation.cc:299] Negotiation successful
0807 08:51:52.319240 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260807 08:51:52.322997 5411 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d523076f2aa04b67904e5d8519bf1dea: Generated new TSK 0
I20260807 08:51:52.323179 5411 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:51:52.327463 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 5274
2026-08-07T08:51:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1362 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786092453486433-4734-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:51:52 dist-test-slave-6zbq krb5kdc[5442](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:51:52 dist-test-slave-6zbq krb5kdc[5442](info): set up 2 sockets
Aug 07 08:51:52 dist-test-slave-6zbq krb5kdc[5442](info): commencing operation
krb5kdc: starting...
W20260807 08:51:53.451704 4734 mini_kdc.cc:121] Time spent starting KDC: real 1.095s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786092453486433-4734-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786092453486433-4734-0/krb5kdc/test-user.keytab.
Aug 07 08:51:53 dist-test-slave-6zbq krb5kdc[5442](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092713, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:51:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:51:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.159.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.159.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.159.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab.
Entry for principal kudu/127.4.159.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab.
I20260807 08:51:53.558040 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786092453486433-4734-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786092453486433-4734-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786092453486433-4734-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.159.190:35123
--webserver_interface=127.4.159.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab
--principal=kudu/127.4.159.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:43205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.159.190:35123
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:51:53.635380 5461 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:51:53.635563 5461 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:51:53.635596 5461 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:51:53.636703 5461 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:51:53.636744 5461 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:51:53.636763 5461 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:51:53.636778 5461 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:51:53.636806 5461 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:51:53.638249 5461 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:43205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786092453486433-4734-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786092453486433-4734-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.159.190:35123
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab
--principal=kudu/127.4.159.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.159.190:35123
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.159.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5461
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786092453486433-4734-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:51:53.638502 5461 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:51:53.638705 5461 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:51:53.643214 5474 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:51:53.643273 5469 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:51:53.643273 5470 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:51:53.643327 5461 server_base.cc:1034] running on GCE node
I20260807 08:51:53.643678 5461 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:51:53.643949 5461 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:51:53.645092 5461 hybrid_clock.cc:648] HybridClock initialized: now 1786092713645074 us; error 30 us; skew 500 ppm
Aug 07 08:51:53 dist-test-slave-6zbq krb5kdc[5442](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092713, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:51:53.647392 5461 init.cc:377] Logged in from keytab as kudu/127.4.159.190@KRBTEST.COM (short username kudu)
I20260807 08:51:53.647802 5461 webserver.cc:460] Webserver started at http://127.4.159.190:34961/ using document root <none> and password file <none>
I20260807 08:51:53.648003 5461 fs_manager.cc:359] Metadata directory not provided
I20260807 08:51:53.648051 5461 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:51:53.648175 5461 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:51:53.649158 5461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786092453486433-4734-0/minicluster-data/master-0/data/instance:
uuid: "521962821d4546618436414ca86f9462"
format_stamp: "Formatted at 2026-08-07 08:51:53 on dist-test-slave-6zbq"
server_key: "0e18818c169c8aae24b30601abfd9f08"
server_key_iv: "47d55d4454e8fec16e048bd06037b553"
server_key_version: "encryptionkey@0"
I20260807 08:51:53.649473 5461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786092453486433-4734-0/minicluster-data/master-0/wal/instance:
uuid: "521962821d4546618436414ca86f9462"
format_stamp: "Formatted at 2026-08-07 08:51:53 on dist-test-slave-6zbq"
server_key: "0e18818c169c8aae24b30601abfd9f08"
server_key_iv: "47d55d4454e8fec16e048bd06037b553"
server_key_version: "encryptionkey@0"
I20260807 08:51:53.650713 5461 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 08:51:53.651417 5484 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:51:53.651602 5461 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 08:51:53.651680 5461 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786092453486433-4734-0/minicluster-data/master-0/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786092453486433-4734-0/minicluster-data/master-0/wal
uuid: "521962821d4546618436414ca86f9462"
format_stamp: "Formatted at 2026-08-07 08:51:53 on dist-test-slave-6zbq"
server_key: "0e18818c169c8aae24b30601abfd9f08"
server_key_iv: "47d55d4454e8fec16e048bd06037b553"
server_key_version: "encryptionkey@0"
I20260807 08:51:53.651767 5461 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786092453486433-4734-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786092453486433-4734-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786092453486433-4734-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:51:53.673416 5461 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:51:53.675463 5461 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:51:53.675599 5461 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:51:53.679368 5461 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.190:35123
I20260807 08:51:53.679404 5564 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.190:35123 every 8 connection(s)
I20260807 08:51:53.679705 5461 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
I20260807 08:51:53.680517 5565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:51:53.681970 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 5461
I20260807 08:51:53.682085 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786092453486433-4734-0/minicluster-data/master-0/wal/instance
I20260807 08:51:53.682273 4734 external_mini_cluster.cc:1428] Setting key 2432aba63cb6a0840e992c2b81d7b522
I20260807 08:51:53.683020 5565 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 521962821d4546618436414ca86f9462: Bootstrap starting.
I20260807 08:51:53.683588 5565 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 521962821d4546618436414ca86f9462: Neither blocks nor log segments found. Creating new log.
I20260807 08:51:53.683910 5565 log.cc:824] T 00000000000000000000000000000000 P 521962821d4546618436414ca86f9462: Log is configured to *not* fsync() on all Append() calls
I20260807 08:51:53.684592 5565 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 521962821d4546618436414ca86f9462: No bootstrap required, opened a new log
I20260807 08:51:53.686637 5565 raft_consensus.cc:348] T 00000000000000000000000000000000 P 521962821d4546618436414ca86f9462 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "521962821d4546618436414ca86f9462" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 35123 } }
I20260807 08:51:53.686781 5565 raft_consensus.cc:374] T 00000000000000000000000000000000 P 521962821d4546618436414ca86f9462 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:51:53.686818 5565 raft_consensus.cc:729] T 00000000000000000000000000000000 P 521962821d4546618436414ca86f9462 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 521962821d4546618436414ca86f9462, State: Initialized, Role: FOLLOWER
I20260807 08:51:53.686944 5565 consensus_queue.cc:260] T 00000000000000000000000000000000 P 521962821d4546618436414ca86f9462 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "521962821d4546618436414ca86f9462" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 35123 } }
I20260807 08:51:53.687031 5565 raft_consensus.cc:388] T 00000000000000000000000000000000 P 521962821d4546618436414ca86f9462 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:51:53.687073 5565 raft_consensus.cc:482] T 00000000000000000000000000000000 P 521962821d4546618436414ca86f9462 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:51:53.687120 5565 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 521962821d4546618436414ca86f9462 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:51:53.687750 5565 raft_consensus.cc:504] T 00000000000000000000000000000000 P 521962821d4546618436414ca86f9462 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "521962821d4546618436414ca86f9462" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 35123 } }
I20260807 08:51:53.687865 5565 leader_election.cc:302] T 00000000000000000000000000000000 P 521962821d4546618436414ca86f9462 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 521962821d4546618436414ca86f9462; no voters:
I20260807 08:51:53.688017 5565 leader_election.cc:288] T 00000000000000000000000000000000 P 521962821d4546618436414ca86f9462 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:51:53.688055 5571 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 521962821d4546618436414ca86f9462 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:51:53.688189 5571 raft_consensus.cc:686] T 00000000000000000000000000000000 P 521962821d4546618436414ca86f9462 [term 1 LEADER]: Becoming Leader. State: Replica: 521962821d4546618436414ca86f9462, State: Running, Role: LEADER
I20260807 08:51:53.688257 5565 sys_catalog.cc:564] T 00000000000000000000000000000000 P 521962821d4546618436414ca86f9462 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:51:53.688328 5571 consensus_queue.cc:237] T 00000000000000000000000000000000 P 521962821d4546618436414ca86f9462 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "521962821d4546618436414ca86f9462" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 35123 } }
I20260807 08:51:53.688657 5571 sys_catalog.cc:455] T 00000000000000000000000000000000 P 521962821d4546618436414ca86f9462 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 521962821d4546618436414ca86f9462. Latest consensus state: current_term: 1 leader_uuid: "521962821d4546618436414ca86f9462" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "521962821d4546618436414ca86f9462" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 35123 } } }
Aug 07 08:51:53 dist-test-slave-6zbq krb5kdc[5442](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092713, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.159.190@KRBTEST.COM
I20260807 08:51:53.688804 5573 sys_catalog.cc:455] T 00000000000000000000000000000000 P 521962821d4546618436414ca86f9462 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "521962821d4546618436414ca86f9462" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "521962821d4546618436414ca86f9462" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 35123 } } }
I20260807 08:51:53.688985 5573 sys_catalog.cc:458] T 00000000000000000000000000000000 P 521962821d4546618436414ca86f9462 [sys.catalog]: This master's current role is: LEADER
I20260807 08:51:53.688867 5571 sys_catalog.cc:458] T 00000000000000000000000000000000 P 521962821d4546618436414ca86f9462 [sys.catalog]: This master's current role is: LEADER
I20260807 08:51:53.689525 5583 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:51:53.690083 5583 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:51:53.691766 5583 catalog_manager.cc:1347] Generated new cluster ID: 702d312abca44d5bad0587197a3db0f2
I20260807 08:51:53.691818 5583 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:51:53.692662 5568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:53.682782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37183 (local address 127.4.159.190:35123)
0807 08:51:53.682937 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:53.682945 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:53.682969 (+ 24us) server_negotiation.cc:407] Connection header received
0807 08:51:53.683403 (+ 434us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:53.683410 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:53.683456 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:53.683566 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:51:53.684313 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:53.685237 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:53.686270 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:53.686675 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:53.686750 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:53.689409 (+ 2659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:51:53.689414 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:51:53.689414 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:51:53.689426 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:51:53.691290 (+ 1864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:51:53.691596 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:51:53.691599 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:51:53.691599 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:51:53.691707 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:51:53.691901 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:51:53.691902 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:51:53.691903 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:51:53.692117 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:51:53.692250 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:53.692375 (+ 125us) server_negotiation.cc:299] Negotiation successful
0807 08:51:53.692505 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":65,"threads_started":1}
I20260807 08:51:53.693441 4734 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 08:51:53.703541 5568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:53.694674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60957 (local address 127.4.159.190:35123)
0807 08:51:53.694707 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:53.694708 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:53.698506 (+ 3798us) server_negotiation.cc:407] Connection header received
0807 08:51:53.698558 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:53.698559 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:53.698568 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:53.698611 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:51:53.699049 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:53.699517 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:53.700143 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:53.700364 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:53.700416 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:53.701413 (+ 997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:51:53.701415 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:51:53.701415 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:51:53.701421 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:51:53.702608 (+ 1187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:51:53.702900 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:51:53.702901 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:51:53.702901 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:51:53.702953 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:51:53.703115 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:51:53.703115 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:51:53.703116 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:51:53.703214 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:51:53.703297 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:53.703393 (+ 96us) server_negotiation.cc:299] Negotiation successful
0807 08:51:53.703461 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 08:51:53.726616 5583 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:51:53.727293 5583 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:51:53.740996 5583 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 521962821d4546618436414ca86f9462: Generated new TSK 0
I20260807 08:51:53.741184 5583 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:51:53.745775 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 5461
2026-08-07T08:51:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1410 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786092453486433-4734-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:51:53 dist-test-slave-6zbq krb5kdc[5609](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:51:53 dist-test-slave-6zbq krb5kdc[5609](info): set up 2 sockets
Aug 07 08:51:53 dist-test-slave-6zbq krb5kdc[5609](info): commencing operation
krb5kdc: starting...
W20260807 08:51:54.855062 4734 mini_kdc.cc:121] Time spent starting KDC: real 1.088s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786092453486433-4734-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786092453486433-4734-0/krb5kdc/test-user.keytab.
Aug 07 08:51:54 dist-test-slave-6zbq krb5kdc[5609](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092714, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:51:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:51:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.159.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.159.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.159.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab.
Entry for principal kudu/127.4.159.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab.
I20260807 08:51:54.961621 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786092453486433-4734-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786092453486433-4734-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786092453486433-4734-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.159.190:44835
--webserver_interface=127.4.159.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab
--principal=kudu/127.4.159.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:39623
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.159.190:44835
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:51:55.040868 5626 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:51:55.041056 5626 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:51:55.041102 5626 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:51:55.042210 5626 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:51:55.042251 5626 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:51:55.042270 5626 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:51:55.042286 5626 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:51:55.042302 5626 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:51:55.043776 5626 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:39623
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786092453486433-4734-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786092453486433-4734-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.159.190:44835
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab
--principal=kudu/127.4.159.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.159.190:44835
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.159.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5626
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786092453486433-4734-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:51:55.044008 5626 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:51:55.044198 5626 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:51:55.048563 5633 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:51:55.048568 5636 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:51:55.048585 5634 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:51:55.048907 5626 server_base.cc:1034] running on GCE node
I20260807 08:51:55.049122 5626 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:51:55.049366 5626 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:51:55.050493 5626 hybrid_clock.cc:648] HybridClock initialized: now 1786092715050482 us; error 34 us; skew 500 ppm
Aug 07 08:51:55 dist-test-slave-6zbq krb5kdc[5609](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092715, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:51:55.052783 5626 init.cc:377] Logged in from keytab as kudu/127.4.159.190@KRBTEST.COM (short username kudu)
I20260807 08:51:55.053175 5626 webserver.cc:460] Webserver started at http://127.4.159.190:36383/ using document root <none> and password file <none>
I20260807 08:51:55.053376 5626 fs_manager.cc:359] Metadata directory not provided
I20260807 08:51:55.053419 5626 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:51:55.053539 5626 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:51:55.054600 5626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786092453486433-4734-0/minicluster-data/master-0/data/instance:
uuid: "dc75a05ff0c4415fadf6deb41b4f13fd"
format_stamp: "Formatted at 2026-08-07 08:51:55 on dist-test-slave-6zbq"
server_key: "b658c15c3c93834fddd081709a55a281"
server_key_iv: "ed361f76fcb854f68c039e02b640f4c9"
server_key_version: "encryptionkey@0"
I20260807 08:51:55.054919 5626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786092453486433-4734-0/minicluster-data/master-0/wal/instance:
uuid: "dc75a05ff0c4415fadf6deb41b4f13fd"
format_stamp: "Formatted at 2026-08-07 08:51:55 on dist-test-slave-6zbq"
server_key: "b658c15c3c93834fddd081709a55a281"
server_key_iv: "ed361f76fcb854f68c039e02b640f4c9"
server_key_version: "encryptionkey@0"
I20260807 08:51:55.056087 5626 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 08:51:55.056696 5644 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:51:55.056845 5626 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 08:51:55.056921 5626 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786092453486433-4734-0/minicluster-data/master-0/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786092453486433-4734-0/minicluster-data/master-0/wal
uuid: "dc75a05ff0c4415fadf6deb41b4f13fd"
format_stamp: "Formatted at 2026-08-07 08:51:55 on dist-test-slave-6zbq"
server_key: "b658c15c3c93834fddd081709a55a281"
server_key_iv: "ed361f76fcb854f68c039e02b640f4c9"
server_key_version: "encryptionkey@0"
I20260807 08:51:55.056983 5626 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786092453486433-4734-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786092453486433-4734-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786092453486433-4734-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:51:55.082597 5626 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:51:55.084517 5626 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:51:55.084640 5626 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:51:55.088358 5626 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.190:44835
I20260807 08:51:55.088393 5719 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.190:44835 every 8 connection(s)
I20260807 08:51:55.088682 5626 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
I20260807 08:51:55.089501 5722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:51:55.091717 5722 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc75a05ff0c4415fadf6deb41b4f13fd: Bootstrap starting.
I20260807 08:51:55.092229 5722 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dc75a05ff0c4415fadf6deb41b4f13fd: Neither blocks nor log segments found. Creating new log.
I20260807 08:51:55.092455 5722 log.cc:824] T 00000000000000000000000000000000 P dc75a05ff0c4415fadf6deb41b4f13fd: Log is configured to *not* fsync() on all Append() calls
I20260807 08:51:55.093045 5722 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc75a05ff0c4415fadf6deb41b4f13fd: No bootstrap required, opened a new log
I20260807 08:51:55.094614 5722 raft_consensus.cc:348] T 00000000000000000000000000000000 P dc75a05ff0c4415fadf6deb41b4f13fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc75a05ff0c4415fadf6deb41b4f13fd" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 44835 } }
I20260807 08:51:55.094741 5722 raft_consensus.cc:374] T 00000000000000000000000000000000 P dc75a05ff0c4415fadf6deb41b4f13fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:51:55.094761 5722 raft_consensus.cc:729] T 00000000000000000000000000000000 P dc75a05ff0c4415fadf6deb41b4f13fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc75a05ff0c4415fadf6deb41b4f13fd, State: Initialized, Role: FOLLOWER
I20260807 08:51:55.094866 5722 consensus_queue.cc:260] T 00000000000000000000000000000000 P dc75a05ff0c4415fadf6deb41b4f13fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc75a05ff0c4415fadf6deb41b4f13fd" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 44835 } }
I20260807 08:51:55.094937 5722 raft_consensus.cc:388] T 00000000000000000000000000000000 P dc75a05ff0c4415fadf6deb41b4f13fd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:51:55.094965 5722 raft_consensus.cc:482] T 00000000000000000000000000000000 P dc75a05ff0c4415fadf6deb41b4f13fd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:51:55.094996 5722 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dc75a05ff0c4415fadf6deb41b4f13fd [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:51:55.095731 5722 raft_consensus.cc:504] T 00000000000000000000000000000000 P dc75a05ff0c4415fadf6deb41b4f13fd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc75a05ff0c4415fadf6deb41b4f13fd" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 44835 } }
I20260807 08:51:55.095818 5722 leader_election.cc:302] T 00000000000000000000000000000000 P dc75a05ff0c4415fadf6deb41b4f13fd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dc75a05ff0c4415fadf6deb41b4f13fd; no voters:
I20260807 08:51:55.095934 5722 leader_election.cc:288] T 00000000000000000000000000000000 P dc75a05ff0c4415fadf6deb41b4f13fd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:51:55.095976 5725 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dc75a05ff0c4415fadf6deb41b4f13fd [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:51:55.096112 5725 raft_consensus.cc:686] T 00000000000000000000000000000000 P dc75a05ff0c4415fadf6deb41b4f13fd [term 1 LEADER]: Becoming Leader. State: Replica: dc75a05ff0c4415fadf6deb41b4f13fd, State: Running, Role: LEADER
I20260807 08:51:55.096127 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 5626
I20260807 08:51:55.096174 5722 sys_catalog.cc:564] T 00000000000000000000000000000000 P dc75a05ff0c4415fadf6deb41b4f13fd [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:51:55.096240 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786092453486433-4734-0/minicluster-data/master-0/wal/instance
I20260807 08:51:55.096289 5725 consensus_queue.cc:237] T 00000000000000000000000000000000 P dc75a05ff0c4415fadf6deb41b4f13fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc75a05ff0c4415fadf6deb41b4f13fd" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 44835 } }
I20260807 08:51:55.096410 4734 external_mini_cluster.cc:1428] Setting key 9c72eb7616b9a965f7faab5ab07f88ab
I20260807 08:51:55.096710 5727 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc75a05ff0c4415fadf6deb41b4f13fd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dc75a05ff0c4415fadf6deb41b4f13fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc75a05ff0c4415fadf6deb41b4f13fd" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 44835 } } }
I20260807 08:51:55.096851 5727 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc75a05ff0c4415fadf6deb41b4f13fd [sys.catalog]: This master's current role is: LEADER
I20260807 08:51:55.097471 5726 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc75a05ff0c4415fadf6deb41b4f13fd [sys.catalog]: SysCatalogTable state changed. Reason: New leader dc75a05ff0c4415fadf6deb41b4f13fd. Latest consensus state: current_term: 1 leader_uuid: "dc75a05ff0c4415fadf6deb41b4f13fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc75a05ff0c4415fadf6deb41b4f13fd" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 44835 } } }
I20260807 08:51:55.097616 5726 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc75a05ff0c4415fadf6deb41b4f13fd [sys.catalog]: This master's current role is: LEADER
I20260807 08:51:55.097476 5739 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:51:55.098315 5739 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:51:55.099952 5739 catalog_manager.cc:1347] Generated new cluster ID: 7c64c9430bc14583b260ad4292ba6fa7
I20260807 08:51:55.100008 5739 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:51:55.101783 5741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:55.096973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32903 (local address 127.4.159.190:44835)
0807 08:51:55.097208 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:55.097216 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:55.097306 (+ 90us) server_negotiation.cc:407] Connection header received
0807 08:51:55.097466 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:55.097473 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:55.097528 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:55.097675 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:51:55.097965 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:55.099224 (+ 1259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:55.100142 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:55.100547 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:55.100646 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:55.101201 (+ 555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:51:55.101205 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:51:55.101205 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:51:55.101370 (+ 165us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:51:55.101399 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:51:55.101409 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:51:55.101463 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:55.101533 (+ 70us) server_negotiation.cc:299] Negotiation successful
0807 08:51:55.101592 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":189,"thread_start_us":75,"threads_started":1}
I20260807 08:51:55.102584 4734 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 08:51:55.108807 5741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:55.103786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46523 (local address 127.4.159.190:44835)
0807 08:51:55.103817 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:55.103819 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:55.106523 (+ 2704us) server_negotiation.cc:407] Connection header received
0807 08:51:55.106573 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:55.106575 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:55.106584 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:55.106627 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:51:55.106743 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:55.107203 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:55.107839 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:55.108030 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:55.108081 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:55.108602 (+ 521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:51:55.108603 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:51:55.108603 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:51:55.108607 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:51:55.108619 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:51:55.108623 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:51:55.108663 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:55.108712 (+ 49us) server_negotiation.cc:299] Negotiation successful
0807 08:51:55.108740 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260807 08:51:55.115176 5739 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:51:55.115619 5739 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:51:55.123693 5739 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dc75a05ff0c4415fadf6deb41b4f13fd: Generated new TSK 0
I20260807 08:51:55.123855 5739 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:51:55.133605 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 5626
2026-08-07T08:51:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1393 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786092453486433-4734-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:51:55 dist-test-slave-6zbq krb5kdc[5766](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:51:55 dist-test-slave-6zbq krb5kdc[5766](info): set up 2 sockets
Aug 07 08:51:55 dist-test-slave-6zbq krb5kdc[5766](info): commencing operation
krb5kdc: starting...
W20260807 08:51:56.250936 4734 mini_kdc.cc:121] Time spent starting KDC: real 1.091s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786092453486433-4734-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786092453486433-4734-0/krb5kdc/test-user.keytab.
Aug 07 08:51:56 dist-test-slave-6zbq krb5kdc[5766](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092716, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:51:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:51:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.159.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.159.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.159.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab.
Entry for principal kudu/127.4.159.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab.
I20260807 08:51:56.356379 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786092453486433-4734-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786092453486433-4734-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786092453486433-4734-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.159.190:42891
--webserver_interface=127.4.159.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab
--principal=kudu/127.4.159.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:42001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.159.190:42891
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:51:56.435726 5785 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:51:56.435913 5785 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:51:56.435943 5785 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:51:56.437028 5785 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:51:56.437067 5785 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:51:56.437086 5785 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:51:56.437100 5785 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:51:56.437130 5785 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:51:56.438566 5785 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:42001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786092453486433-4734-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786092453486433-4734-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.159.190:42891
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab
--principal=kudu/127.4.159.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.159.190:42891
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.159.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5785
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786092453486433-4734-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:51:56.438786 5785 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:51:56.438967 5785 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:51:56.443289 5794 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:51:56.443306 5798 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:51:56.443306 5801 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:51:56.443562 5785 server_base.cc:1034] running on GCE node
I20260807 08:51:56.443787 5785 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:51:56.444065 5785 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:51:56.445241 5785 hybrid_clock.cc:648] HybridClock initialized: now 1786092716445219 us; error 33 us; skew 500 ppm
Aug 07 08:51:56 dist-test-slave-6zbq krb5kdc[5766](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092716, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:51:56.451936 5785 init.cc:377] Logged in from keytab as kudu/127.4.159.190@KRBTEST.COM (short username kudu)
I20260807 08:51:56.452340 5785 webserver.cc:460] Webserver started at http://127.4.159.190:44647/ using document root <none> and password file <none>
I20260807 08:51:56.452530 5785 fs_manager.cc:359] Metadata directory not provided
I20260807 08:51:56.452577 5785 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:51:56.452696 5785 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:51:56.453686 5785 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786092453486433-4734-0/minicluster-data/master-0/data/instance:
uuid: "847e58567bf84ab0a4fc7be08155f8a9"
format_stamp: "Formatted at 2026-08-07 08:51:56 on dist-test-slave-6zbq"
server_key: "d013caa2318071e81c8cbf3797e24f7e"
server_key_iv: "60ae665c0a327ff49934712c4932021d"
server_key_version: "encryptionkey@0"
I20260807 08:51:56.454016 5785 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786092453486433-4734-0/minicluster-data/master-0/wal/instance:
uuid: "847e58567bf84ab0a4fc7be08155f8a9"
format_stamp: "Formatted at 2026-08-07 08:51:56 on dist-test-slave-6zbq"
server_key: "d013caa2318071e81c8cbf3797e24f7e"
server_key_iv: "60ae665c0a327ff49934712c4932021d"
server_key_version: "encryptionkey@0"
I20260807 08:51:56.455291 5785 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 08:51:56.455955 5812 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:51:56.456097 5785 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 08:51:56.456178 5785 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786092453486433-4734-0/minicluster-data/master-0/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786092453486433-4734-0/minicluster-data/master-0/wal
uuid: "847e58567bf84ab0a4fc7be08155f8a9"
format_stamp: "Formatted at 2026-08-07 08:51:56 on dist-test-slave-6zbq"
server_key: "d013caa2318071e81c8cbf3797e24f7e"
server_key_iv: "60ae665c0a327ff49934712c4932021d"
server_key_version: "encryptionkey@0"
I20260807 08:51:56.456236 5785 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786092453486433-4734-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786092453486433-4734-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786092453486433-4734-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:51:56.459477 5785 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:51:56.461225 5785 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:51:56.461357 5785 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:51:56.465197 5785 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.190:42891
I20260807 08:51:56.465227 5882 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.190:42891 every 8 connection(s)
I20260807 08:51:56.465516 5785 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
I20260807 08:51:56.466449 5884 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:51:56.468537 5884 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 847e58567bf84ab0a4fc7be08155f8a9: Bootstrap starting.
I20260807 08:51:56.469076 5884 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 847e58567bf84ab0a4fc7be08155f8a9: Neither blocks nor log segments found. Creating new log.
I20260807 08:51:56.469336 5884 log.cc:824] T 00000000000000000000000000000000 P 847e58567bf84ab0a4fc7be08155f8a9: Log is configured to *not* fsync() on all Append() calls
I20260807 08:51:56.469916 5884 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 847e58567bf84ab0a4fc7be08155f8a9: No bootstrap required, opened a new log
I20260807 08:51:56.470525 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 5785
I20260807 08:51:56.470620 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786092453486433-4734-0/minicluster-data/master-0/wal/instance
I20260807 08:51:56.470813 4734 external_mini_cluster.cc:1428] Setting key fa39e0881baa5bc236a6951dbdc86554
I20260807 08:51:56.471899 5884 raft_consensus.cc:348] T 00000000000000000000000000000000 P 847e58567bf84ab0a4fc7be08155f8a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "847e58567bf84ab0a4fc7be08155f8a9" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 42891 } }
I20260807 08:51:56.472059 5884 raft_consensus.cc:374] T 00000000000000000000000000000000 P 847e58567bf84ab0a4fc7be08155f8a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:51:56.472095 5884 raft_consensus.cc:729] T 00000000000000000000000000000000 P 847e58567bf84ab0a4fc7be08155f8a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 847e58567bf84ab0a4fc7be08155f8a9, State: Initialized, Role: FOLLOWER
I20260807 08:51:56.472471 5884 consensus_queue.cc:260] T 00000000000000000000000000000000 P 847e58567bf84ab0a4fc7be08155f8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "847e58567bf84ab0a4fc7be08155f8a9" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 42891 } }
I20260807 08:51:56.472575 5884 raft_consensus.cc:388] T 00000000000000000000000000000000 P 847e58567bf84ab0a4fc7be08155f8a9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:51:56.472609 5884 raft_consensus.cc:482] T 00000000000000000000000000000000 P 847e58567bf84ab0a4fc7be08155f8a9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:51:56.472657 5884 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 847e58567bf84ab0a4fc7be08155f8a9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:51:56.473011 5891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:56.471265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38541 (local address 127.4.159.190:42891)
0807 08:51:56.471442 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:56.471451 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:56.471486 (+ 35us) server_negotiation.cc:407] Connection header received
0807 08:51:56.471870 (+ 384us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:56.471878 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:56.471911 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:56.472058 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:51:56.472578 (+ 520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:51:56.472581 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:51:56.472582 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:51:56.472586 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:51:56.472632 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:51:56.472647 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:51:56.472715 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:56.472777 (+ 62us) server_negotiation.cc:299] Negotiation successful
0807 08:51:56.472823 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":75,"threads_started":1}
I20260807 08:51:56.473552 5884 raft_consensus.cc:504] T 00000000000000000000000000000000 P 847e58567bf84ab0a4fc7be08155f8a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "847e58567bf84ab0a4fc7be08155f8a9" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 42891 } }
I20260807 08:51:56.473637 5884 leader_election.cc:302] T 00000000000000000000000000000000 P 847e58567bf84ab0a4fc7be08155f8a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 847e58567bf84ab0a4fc7be08155f8a9; no voters:
I20260807 08:51:56.473770 5884 leader_election.cc:288] T 00000000000000000000000000000000 P 847e58567bf84ab0a4fc7be08155f8a9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:51:56.473814 5893 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 847e58567bf84ab0a4fc7be08155f8a9 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:51:56.473905 4734 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 08:51:56.473948 5893 raft_consensus.cc:686] T 00000000000000000000000000000000 P 847e58567bf84ab0a4fc7be08155f8a9 [term 1 LEADER]: Becoming Leader. State: Replica: 847e58567bf84ab0a4fc7be08155f8a9, State: Running, Role: LEADER
I20260807 08:51:56.474020 5884 sys_catalog.cc:564] T 00000000000000000000000000000000 P 847e58567bf84ab0a4fc7be08155f8a9 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:51:56.474025 5893 consensus_queue.cc:237] T 00000000000000000000000000000000 P 847e58567bf84ab0a4fc7be08155f8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "847e58567bf84ab0a4fc7be08155f8a9" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 42891 } }
I20260807 08:51:56.474403 5894 sys_catalog.cc:455] T 00000000000000000000000000000000 P 847e58567bf84ab0a4fc7be08155f8a9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "847e58567bf84ab0a4fc7be08155f8a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "847e58567bf84ab0a4fc7be08155f8a9" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 42891 } } }
I20260807 08:51:56.474480 5894 sys_catalog.cc:458] T 00000000000000000000000000000000 P 847e58567bf84ab0a4fc7be08155f8a9 [sys.catalog]: This master's current role is: LEADER
I20260807 08:51:56.474625 5895 sys_catalog.cc:455] T 00000000000000000000000000000000 P 847e58567bf84ab0a4fc7be08155f8a9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 847e58567bf84ab0a4fc7be08155f8a9. Latest consensus state: current_term: 1 leader_uuid: "847e58567bf84ab0a4fc7be08155f8a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "847e58567bf84ab0a4fc7be08155f8a9" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 42891 } } }
I20260807 08:51:56.474686 5895 sys_catalog.cc:458] T 00000000000000000000000000000000 P 847e58567bf84ab0a4fc7be08155f8a9 [sys.catalog]: This master's current role is: LEADER
W20260807 08:51:56.475739 5922 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 847e58567bf84ab0a4fc7be08155f8a9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 08:51:56.475835 5922 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 08:51:56.475906 5901 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:51:56.476046 5901 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:51:56.476488 5891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:56.475534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59115 (local address 127.4.159.190:42891)
0807 08:51:56.475564 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:56.475565 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:56.475687 (+ 122us) server_negotiation.cc:407] Connection header received
0807 08:51:56.475734 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:56.475735 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:56.475739 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:56.475792 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:51:56.476277 (+ 485us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:51:56.476278 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:51:56.476279 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:51:56.476291 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:51:56.476305 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:51:56.476310 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:51:56.476366 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:56.476423 (+ 57us) server_negotiation.cc:299] Negotiation successful
0807 08:51:56.476435 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260807 08:51:56.477384 5901 catalog_manager.cc:1347] Generated new cluster ID: 101d83d3196b4c40807d28bb2108dfba
I20260807 08:51:56.477437 5901 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:51:56.500663 5901 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:51:56.500828 5901 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:51:56.507066 5901 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 847e58567bf84ab0a4fc7be08155f8a9: Generated new TSK 0
I20260807 08:51:56.507211 5901 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:51:56.515344 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 5785
2026-08-07T08:51:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1377 ms)
[----------] 18 tests from AuthTokenIssuingTest (24904 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786092453486433-4734-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:51:56 dist-test-slave-6zbq krb5kdc[5929](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:51:56 dist-test-slave-6zbq krb5kdc[5929](info): set up 2 sockets
Aug 07 08:51:56 dist-test-slave-6zbq krb5kdc[5929](info): commencing operation
krb5kdc: starting...
W20260807 08:51:57.639883 4734 mini_kdc.cc:121] Time spent starting KDC: real 1.102s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786092453486433-4734-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786092453486433-4734-0/krb5kdc/test-user.keytab.
Aug 07 08:51:57 dist-test-slave-6zbq krb5kdc[5929](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092717, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:51:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:51:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.159.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.159.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.159.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab.
Entry for principal kudu/127.4.159.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab.
I20260807 08:51:57.748301 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786092453486433-4734-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786092453486433-4734-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786092453486433-4734-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.159.190:36953
--webserver_interface=127.4.159.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab
--principal=kudu/127.4.159.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:40675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.159.190:36953,127.4.159.189:40563,127.4.159.188:44353
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:51:57.827649 5946 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 08:51:57.827840 5946 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:51:57.827864 5946 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:51:57.827888 5946 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:51:57.828961 5946 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:51:57.829000 5946 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:51:57.829020 5946 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:51:57.829034 5946 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:51:57.829051 5946 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:51:57.830565 5946 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:40675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786092453486433-4734-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786092453486433-4734-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.159.190:36953,127.4.159.189:40563,127.4.159.188:44353
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab
--principal=kudu/127.4.159.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.159.190:36953
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.159.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5946
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786092453486433-4734-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:51:57.830799 5946 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:51:57.830992 5946 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:51:57.835470 5955 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:51:57.835458 5956 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:51:57.835608 5946 server_base.cc:1034] running on GCE node
W20260807 08:51:57.835446 5961 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:51:57.835881 5946 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:51:57.836136 5946 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:51:57.837278 5946 hybrid_clock.cc:648] HybridClock initialized: now 1786092717837265 us; error 30 us; skew 500 ppm
Aug 07 08:51:57 dist-test-slave-6zbq krb5kdc[5929](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092717, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:51:57.839532 5946 init.cc:377] Logged in from keytab as kudu/127.4.159.190@KRBTEST.COM (short username kudu)
I20260807 08:51:57.839902 5946 webserver.cc:460] Webserver started at http://127.4.159.190:40097/ using document root <none> and password file <none>
I20260807 08:51:57.840088 5946 fs_manager.cc:359] Metadata directory not provided
I20260807 08:51:57.840126 5946 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:51:57.840214 5946 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:51:57.841161 5946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786092453486433-4734-0/minicluster-data/master-0/data/instance:
uuid: "67b1b2da5df841f89e9c42793cb37b1a"
format_stamp: "Formatted at 2026-08-07 08:51:57 on dist-test-slave-6zbq"
server_key: "f816ca8083c9da23aae07f856e928118"
server_key_iv: "11777244adfdfbe5c43b82fca26b2f03"
server_key_version: "encryptionkey@0"
I20260807 08:51:57.841446 5946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786092453486433-4734-0/minicluster-data/master-0/wal/instance:
uuid: "67b1b2da5df841f89e9c42793cb37b1a"
format_stamp: "Formatted at 2026-08-07 08:51:57 on dist-test-slave-6zbq"
server_key: "f816ca8083c9da23aae07f856e928118"
server_key_iv: "11777244adfdfbe5c43b82fca26b2f03"
server_key_version: "encryptionkey@0"
I20260807 08:51:57.842630 5946 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 08:51:57.843297 5970 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:51:57.843441 5946 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 08:51:57.843513 5946 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786092453486433-4734-0/minicluster-data/master-0/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786092453486433-4734-0/minicluster-data/master-0/wal
uuid: "67b1b2da5df841f89e9c42793cb37b1a"
format_stamp: "Formatted at 2026-08-07 08:51:57 on dist-test-slave-6zbq"
server_key: "f816ca8083c9da23aae07f856e928118"
server_key_iv: "11777244adfdfbe5c43b82fca26b2f03"
server_key_version: "encryptionkey@0"
I20260807 08:51:57.843575 5946 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786092453486433-4734-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786092453486433-4734-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786092453486433-4734-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:51:57.858745 5946 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:51:57.860477 5946 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:51:57.860603 5946 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:51:57.864426 5946 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.190:36953
I20260807 08:51:57.864466 6044 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.190:36953 every 8 connection(s)
I20260807 08:51:57.864777 5946 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
I20260807 08:51:57.865680 6045 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:51:57.867161 6045 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.159.190" port: 36953 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 08:51:57 dist-test-slave-6zbq krb5kdc[5929](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092717, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.190@KRBTEST.COM for kudu/127.4.159.190@KRBTEST.COM
I20260807 08:51:57.872575 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 5946
I20260807 08:51:57.872677 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786092453486433-4734-0/minicluster-data/master-0/wal/instance
I20260807 08:51:57.872833 4734 external_mini_cluster.cc:1428] Setting key d23ce0aaa9e3f00980ca55af44b8ab32
I20260807 08:51:57.875562 6050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:57.867775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44190 (local address 127.4.159.190:36953)
0807 08:51:57.867925 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:57.867929 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:57.867998 (+ 69us) server_negotiation.cc:407] Connection header received
0807 08:51:57.868466 (+ 468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:57.868467 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:57.868484 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:57.868552 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:51:57.869010 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:57.869783 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:57.870639 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:57.870934 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:57.871004 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:57.872851 (+ 1847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:51:57.872859 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:51:57.872859 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:51:57.872865 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:51:57.874121 (+ 1256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:51:57.874570 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:51:57.874573 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:51:57.874574 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:51:57.874655 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:51:57.874871 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:51:57.874872 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:51:57.874872 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:51:57.875045 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:51:57.875102 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:57.875296 (+ 194us) server_negotiation.cc:299] Negotiation successful
0807 08:51:57.875397 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":121,"thread_start_us":73,"threads_started":1}
I20260807 08:51:57.875590 6049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:57.867731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:36953 (local address 127.0.0.1:44190)
0807 08:51:57.867878 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:57.867917 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 08:51:57.868355 (+ 438us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:51:57.868568 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:51:57.868570 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:51:57.868800 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:51:57.868947 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:57.868951 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:57.869850 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:57.869851 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:57.870575 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:57.870577 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:57.871015 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:57.871015 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:57.871063 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:57.871468 (+ 405us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:51:57.871476 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:51:57.872781 (+ 1305us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:51:57.874197 (+ 1416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:51:57.874203 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:51:57.874205 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:51:57.874478 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:51:57.874713 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:51:57.874713 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:51:57.874714 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:51:57.874815 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:51:57.875102 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:51:57.875107 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:51:57.875175 (+ 68us) client_negotiation.cc:769] Sending connection context
0807 08:51:57.875275 (+ 100us) client_negotiation.cc:240] Negotiation successful
0807 08:51:57.875480 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"mutex_wait_us":70,"thread_start_us":61,"threads_started":1}
I20260807 08:51:57.876323 6045 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.159.189" port: 40563 } has no permanent_uuid. Determining permanent_uuid...
I20260807 08:51:57.876600 6049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:57.876500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.189:40563 (local address 127.0.0.1:38896)
0807 08:51:57.876532 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:57.876579 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.189:40563: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
W20260807 08:51:57.876731 5974 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.159.189:40563: connect: Connection refused (error 111)
I20260807 08:51:57.877115 6049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:57.877023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.189:40563 (local address 127.0.0.1:38898)
0807 08:51:57.877066 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:57.877094 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.189:40563: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260807 08:51:57.877267 6045 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.159.189:40563: Network error: Client connection negotiation failed: client connection to 127.4.159.189:40563: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.4.159.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.159.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.159.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.189.keytab.
Entry for principal kudu/127.4.159.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.189.keytab.
I20260807 08:51:57.894055 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786092453486433-4734-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786092453486433-4734-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786092453486433-4734-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786092453486433-4734-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.159.189:40563
--webserver_interface=127.4.159.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.189.keytab
--principal=kudu/127.4.159.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:40675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.159.190:36953,127.4.159.189:40563,127.4.159.188:44353
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 08:51:57.926522 6045 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.159.189" port: 40563 } attempt: 1
I20260807 08:51:57.927026 6049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:57.926879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.189:40563 (local address 127.0.0.1:38904)
0807 08:51:57.926946 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:57.926992 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.189:40563: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260807 08:51:57.927368 6049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:57.927286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.189:40563 (local address 127.0.0.1:38906)
0807 08:51:57.927314 (+ 28us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:57.927345 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.189:40563: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
W20260807 08:51:57.927479 6045 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.159.189:40563: Network error: Client connection negotiation failed: client connection to 127.4.159.189:40563: connect: Connection refused (error 111)
W20260807 08:51:57.974113 6058 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 08:51:57.974303 6058 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:51:57.974319 6058 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:51:57.974357 6058 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:51:57.975454 6058 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:51:57.975492 6058 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:51:57.975504 6058 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:51:57.975528 6058 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:51:57.975550 6058 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:51:57.976990 6058 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:40675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786092453486433-4734-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786092453486433-4734-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.4.159.190:36953,127.4.159.189:40563,127.4.159.188:44353
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.189.keytab
--principal=kudu/127.4.159.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.159.189:40563
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786092453486433-4734-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.4.159.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6058
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786092453486433-4734-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:51:57.977208 6058 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:51:57.977406 6058 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 08:51:57.981914 6058 server_base.cc:1034] running on GCE node
W20260807 08:51:57.981907 6064 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:51:57.981889 6067 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:51:57.981890 6065 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:51:57.982267 6058 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:51:57.982573 6058 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:51:57.983708 6058 hybrid_clock.cc:648] HybridClock initialized: now 1786092717983696 us; error 29 us; skew 500 ppm
Aug 07 08:51:57 dist-test-slave-6zbq krb5kdc[5929](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092717, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:51:57.986059 6058 init.cc:377] Logged in from keytab as kudu/127.4.159.189@KRBTEST.COM (short username kudu)
I20260807 08:51:57.986526 6058 webserver.cc:460] Webserver started at http://127.4.159.189:42973/ using document root <none> and password file <none>
I20260807 08:51:57.986698 6058 fs_manager.cc:359] Metadata directory not provided
I20260807 08:51:57.986735 6058 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:51:57.986828 6058 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:51:57.987777 6058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786092453486433-4734-0/minicluster-data/master-1/data/instance:
uuid: "e64815bd02294e9c9dc880fc2fb5fbdf"
format_stamp: "Formatted at 2026-08-07 08:51:57 on dist-test-slave-6zbq"
server_key: "204483edd27c593e1757fbaac1f30e74"
server_key_iv: "8788baa7f2f7cf23320478fe315fcd21"
server_key_version: "encryptionkey@0"
I20260807 08:51:57.988085 6058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786092453486433-4734-0/minicluster-data/master-1/wal/instance:
uuid: "e64815bd02294e9c9dc880fc2fb5fbdf"
format_stamp: "Formatted at 2026-08-07 08:51:57 on dist-test-slave-6zbq"
server_key: "204483edd27c593e1757fbaac1f30e74"
server_key_iv: "8788baa7f2f7cf23320478fe315fcd21"
server_key_version: "encryptionkey@0"
I20260807 08:51:57.989223 6058 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 08:51:57.989836 6076 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:51:57.989985 6058 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 08:51:57.990069 6058 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786092453486433-4734-0/minicluster-data/master-1/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786092453486433-4734-0/minicluster-data/master-1/wal
uuid: "e64815bd02294e9c9dc880fc2fb5fbdf"
format_stamp: "Formatted at 2026-08-07 08:51:57 on dist-test-slave-6zbq"
server_key: "204483edd27c593e1757fbaac1f30e74"
server_key_iv: "8788baa7f2f7cf23320478fe315fcd21"
server_key_version: "encryptionkey@0"
I20260807 08:51:57.990129 6058 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786092453486433-4734-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786092453486433-4734-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786092453486433-4734-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:51:57.995622 6045 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.159.189" port: 40563 } attempt: 2
I20260807 08:51:57.995956 6049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:57.995847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.189:40563 (local address 127.0.0.1:38922)
0807 08:51:57.995899 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:57.995929 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.189:40563: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260807 08:51:57.996279 6049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:57.996187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.189:40563 (local address 127.0.0.1:38928)
0807 08:51:57.996228 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:57.996254 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.189:40563: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260807 08:51:57.996356 6045 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.159.189:40563: Network error: Client connection negotiation failed: client connection to 127.4.159.189:40563: connect: Connection refused (error 111)
I20260807 08:51:58.019604 6058 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:51:58.021561 6058 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:51:58.021687 6058 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:51:58.025370 6058 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.189:40563
I20260807 08:51:58.025409 6149 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.189:40563 every 8 connection(s)
I20260807 08:51:58.025705 6058 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786092453486433-4734-0/minicluster-data/master-1/data/info.pb
I20260807 08:51:58.026623 6152 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:51:58.028025 6152 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.159.190" port: 36953 } has no permanent_uuid. Determining permanent_uuid...
I20260807 08:51:58.028564 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 6058
I20260807 08:51:58.028671 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786092453486433-4734-0/minicluster-data/master-1/wal/instance
I20260807 08:51:58.028879 4734 external_mini_cluster.cc:1428] Setting key 0a6ea9c7f85673143d7dd180ebd9245e
Aug 07 08:51:58 dist-test-slave-6zbq krb5kdc[5929](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092717, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.189@KRBTEST.COM for kudu/127.4.159.190@KRBTEST.COM
I20260807 08:51:58.036798 6050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:58.028650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44206 (local address 127.4.159.190:36953)
0807 08:51:58.028700 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:58.028702 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:58.028888 (+ 186us) server_negotiation.cc:407] Connection header received
0807 08:51:58.029494 (+ 606us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:58.029495 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:58.029509 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:58.029562 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:51:58.030303 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:58.030975 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:58.031917 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:58.032228 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:58.032296 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:58.034282 (+ 1986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:51:58.034283 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:51:58.034284 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:51:58.034291 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:51:58.035565 (+ 1274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:51:58.035921 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:51:58.035922 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:51:58.035923 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:51:58.035977 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:51:58.036188 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:51:58.036189 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:51:58.036189 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:51:58.036304 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:51:58.036361 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:58.036606 (+ 245us) server_negotiation.cc:299] Negotiation successful
0807 08:51:58.036693 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260807 08:51:58.036929 6155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:58.028591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:36953 (local address 127.0.0.1:44206)
0807 08:51:58.028750 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:58.028799 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 08:51:58.029388 (+ 589us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:51:58.029634 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:51:58.029637 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:51:58.029940 (+ 303us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:51:58.030194 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:58.030200 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:58.031067 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:58.031068 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:58.031834 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:58.031836 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:58.032324 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:58.032326 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:58.032433 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:58.032850 (+ 417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:51:58.032858 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:51:58.034180 (+ 1322us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:51:58.035638 (+ 1458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:51:58.035646 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:51:58.035648 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:51:58.035856 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:51:58.036031 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:51:58.036032 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:51:58.036032 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:51:58.036131 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:51:58.036359 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:51:58.036366 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:51:58.036492 (+ 126us) client_negotiation.cc:769] Sending connection context
0807 08:51:58.036577 (+ 85us) client_negotiation.cc:240] Negotiation successful
0807 08:51:58.036717 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":71,"threads_started":1}
WARNING: no policy specified for kudu/127.4.159.188@KRBTEST.COM; defaulting to no policy
I20260807 08:51:58.037643 6152 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.159.189" port: 40563 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.159.188@KRBTEST.COM" created.
Aug 07 08:51:58 dist-test-slave-6zbq krb5kdc[5929](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092717, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.189@KRBTEST.COM for kudu/127.4.159.189@KRBTEST.COM
I20260807 08:51:58.044616 6155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:58.037875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.189:40563 (local address 127.0.0.1:38936)
0807 08:51:58.037915 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:58.037926 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 08:51:58.037965 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:51:58.038309 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:51:58.038310 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:51:58.038447 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:51:58.038487 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:58.038488 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:58.039315 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:58.039316 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:58.039922 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:58.039924 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:58.040399 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:58.040401 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:58.040480 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:58.040797 (+ 317us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:51:58.040802 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:51:58.042102 (+ 1300us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:51:58.043763 (+ 1661us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:51:58.043764 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:51:58.043764 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:51:58.043914 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:51:58.044106 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:51:58.044107 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:51:58.044107 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:51:58.044149 (+ 42us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:51:58.044411 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:51:58.044412 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:51:58.044443 (+ 31us) client_negotiation.cc:769] Sending connection context
0807 08:51:58.044487 (+ 44us) client_negotiation.cc:240] Negotiation successful
0807 08:51:58.044558 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260807 08:51:58.044718 6159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:58.037934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38936 (local address 127.4.159.189:40563)
0807 08:51:58.038174 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:58.038178 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:58.038194 (+ 16us) server_negotiation.cc:407] Connection header received
0807 08:51:58.038211 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:58.038212 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:58.038222 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:58.038284 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:51:58.038549 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:58.039237 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:58.040001 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:58.040326 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:58.040383 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:58.042185 (+ 1802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:51:58.042192 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:51:58.042193 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:51:58.042198 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:51:58.043670 (+ 1472us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:51:58.043984 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:51:58.043987 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:51:58.043988 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:51:58.044047 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:51:58.044185 (+ 138us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:51:58.044186 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:51:58.044186 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:51:58.044352 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:51:58.044437 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:58.044522 (+ 85us) server_negotiation.cc:299] Negotiation successful
0807 08:51:58.044619 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":19,"server-negotiator.queue_time_us":215,"thread_start_us":69,"threads_started":1}
I20260807 08:51:58.045320 6152 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.159.188" port: 44353 } has no permanent_uuid. Determining permanent_uuid...
I20260807 08:51:58.045655 6155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:58.045519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.188:44353 (local address 127.0.0.1:45382)
0807 08:51:58.045559 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:58.045636 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.188:44353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260807 08:51:58.045799 6080 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.159.188:44353: connect: Connection refused (error 111)
I20260807 08:51:58.046182 6155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:58.046104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.188:44353 (local address 127.0.0.1:45386)
0807 08:51:58.046144 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:58.046167 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.188:44353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260807 08:51:58.046296 6152 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.159.188:44353: Network error: Client connection negotiation failed: client connection to 127.4.159.188:44353: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.159.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.188.keytab.
Entry for principal kudu/127.4.159.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.188.keytab.
I20260807 08:51:58.052981 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786092453486433-4734-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786092453486433-4734-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786092453486433-4734-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786092453486433-4734-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.159.188:44353
--webserver_interface=127.4.159.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.188.keytab
--principal=kudu/127.4.159.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:40675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.159.190:36953,127.4.159.189:40563,127.4.159.188:44353
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 08:51:58.087507 6045 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.159.189" port: 40563 } attempt: 3
Aug 07 08:51:58 dist-test-slave-6zbq krb5kdc[5929](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092717, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.190@KRBTEST.COM for kudu/127.4.159.189@KRBTEST.COM
I20260807 08:51:58.094317 6159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:58.087955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38944 (local address 127.4.159.189:40563)
0807 08:51:58.088030 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:58.088032 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:58.088046 (+ 14us) server_negotiation.cc:407] Connection header received
0807 08:51:58.088093 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:58.088094 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:58.088107 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:58.088153 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:51:58.088448 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:58.089220 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:58.089870 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:58.090210 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:58.090279 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:58.092135 (+ 1856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:51:58.092137 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:51:58.092137 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:51:58.092143 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:51:58.093355 (+ 1212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:51:58.093711 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:51:58.093712 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:51:58.093713 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:51:58.093763 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:51:58.093927 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:51:58.093928 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:51:58.093928 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:51:58.094018 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:51:58.094061 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:58.094195 (+ 134us) server_negotiation.cc:299] Negotiation successful
0807 08:51:58.094254 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 08:51:58.094399 6049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:58.087901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.189:40563 (local address 127.0.0.1:38944)
0807 08:51:58.087963 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:58.087976 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:51:58.088035 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:51:58.088173 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:51:58.088173 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:51:58.088317 (+ 144us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:51:58.088380 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:58.088382 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:58.089289 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:58.089290 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:58.089799 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:58.089801 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:58.090286 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:58.090287 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:58.090381 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:58.090797 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:51:58.090801 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:51:58.092030 (+ 1229us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:51:58.093438 (+ 1408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:51:58.093439 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:51:58.093440 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:51:58.093648 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:51:58.093816 (+ 168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:51:58.093817 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:51:58.093818 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:51:58.093879 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:51:58.094064 (+ 185us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:51:58.094065 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:51:58.094113 (+ 48us) client_negotiation.cc:769] Sending connection context
0807 08:51:58.094177 (+ 64us) client_negotiation.cc:240] Negotiation successful
0807 08:51:58.094271 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260807 08:51:58.094983 6045 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.159.188" port: 44353 } has no permanent_uuid. Determining permanent_uuid...
I20260807 08:51:58.095335 6049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:58.095220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.188:44353 (local address 127.0.0.1:45390)
0807 08:51:58.095268 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:58.095309 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.188:44353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260807 08:51:58.095505 6152 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.159.188" port: 44353 } attempt: 1
I20260807 08:51:58.095991 6049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:58.095896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.188:44353 (local address 127.0.0.1:45392)
0807 08:51:58.095940 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:58.095970 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.188:44353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260807 08:51:58.096086 6155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:58.095984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.188:44353 (local address 127.0.0.1:45402)
0807 08:51:58.096031 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:58.096060 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.188:44353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260807 08:51:58.096143 6045 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.159.188:44353: Network error: Client connection negotiation failed: client connection to 127.4.159.188:44353: connect: Connection refused (error 111)
I20260807 08:51:58.096355 6155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:58.096270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.188:44353 (local address 127.0.0.1:45410)
0807 08:51:58.096309 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:58.096333 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.188:44353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260807 08:51:58.096450 6152 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.159.188:44353: Network error: Client connection negotiation failed: client connection to 127.4.159.188:44353: connect: Connection refused (error 111)
I20260807 08:51:58.127272 6045 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.159.188" port: 44353 } attempt: 1
I20260807 08:51:58.127645 6049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:58.127528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.188:44353 (local address 127.0.0.1:45424)
0807 08:51:58.127581 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:58.127616 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.188:44353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260807 08:51:58.127974 6049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:58.127884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.188:44353 (local address 127.0.0.1:45436)
0807 08:51:58.127922 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:58.127950 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.188:44353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260807 08:51:58.128062 6045 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.159.188:44353: Network error: Client connection negotiation failed: client connection to 127.4.159.188:44353: connect: Connection refused (error 111)
W20260807 08:51:58.133701 6164 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 08:51:58.133875 6164 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:51:58.133890 6164 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:51:58.133914 6164 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:51:58.134994 6164 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:51:58.135035 6164 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:51:58.135047 6164 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:51:58.135064 6164 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:51:58.135075 6164 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:51:58.136469 6164 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:40675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786092453486433-4734-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786092453486433-4734-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.4.159.190:36953,127.4.159.189:40563,127.4.159.188:44353
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.188.keytab
--principal=kudu/127.4.159.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.159.188:44353
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786092453486433-4734-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.4.159.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6164
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786092453486433-4734-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:51:58.136651 6164 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:51:58.136844 6164 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:51:58.141088 6169 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:51:58.141134 6171 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:51:58.141088 6173 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:51:58.141232 6164 server_base.cc:1034] running on GCE node
I20260807 08:51:58.141521 6164 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:51:58.141757 6164 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:51:58.142885 6164 hybrid_clock.cc:648] HybridClock initialized: now 1786092718142872 us; error 26 us; skew 500 ppm
Aug 07 08:51:58 dist-test-slave-6zbq krb5kdc[5929](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092718, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:51:58.145081 6164 init.cc:377] Logged in from keytab as kudu/127.4.159.188@KRBTEST.COM (short username kudu)
I20260807 08:51:58.145488 6164 webserver.cc:460] Webserver started at http://127.4.159.188:39597/ using document root <none> and password file <none>
I20260807 08:51:58.145686 6164 fs_manager.cc:359] Metadata directory not provided
I20260807 08:51:58.145731 6164 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:51:58.145860 6164 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:51:58.146914 6164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786092453486433-4734-0/minicluster-data/master-2/data/instance:
uuid: "b85be12e84fa44ad98a556ee5c488a80"
format_stamp: "Formatted at 2026-08-07 08:51:58 on dist-test-slave-6zbq"
server_key: "7cd2bc38334ab64787e04d1e5f306a26"
server_key_iv: "a1b858b7257311edca29617d34843fb1"
server_key_version: "encryptionkey@0"
I20260807 08:51:58.147236 6164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786092453486433-4734-0/minicluster-data/master-2/wal/instance:
uuid: "b85be12e84fa44ad98a556ee5c488a80"
format_stamp: "Formatted at 2026-08-07 08:51:58 on dist-test-slave-6zbq"
server_key: "7cd2bc38334ab64787e04d1e5f306a26"
server_key_iv: "a1b858b7257311edca29617d34843fb1"
server_key_version: "encryptionkey@0"
I20260807 08:51:58.148420 6164 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 08:51:58.149049 6182 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:51:58.149194 6164 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 08:51:58.149286 6164 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786092453486433-4734-0/minicluster-data/master-2/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786092453486433-4734-0/minicluster-data/master-2/wal
uuid: "b85be12e84fa44ad98a556ee5c488a80"
format_stamp: "Formatted at 2026-08-07 08:51:58 on dist-test-slave-6zbq"
server_key: "7cd2bc38334ab64787e04d1e5f306a26"
server_key_iv: "a1b858b7257311edca29617d34843fb1"
server_key_version: "encryptionkey@0"
I20260807 08:51:58.149348 6164 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786092453486433-4734-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786092453486433-4734-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786092453486433-4734-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:51:58.159840 6164 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:51:58.161741 6164 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:51:58.161870 6164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:51:58.164574 6152 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.159.188" port: 44353 } attempt: 2
I20260807 08:51:58.164932 6155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:58.164799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.188:44353 (local address 127.0.0.1:45442)
0807 08:51:58.164850 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:58.164893 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.188:44353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260807 08:51:58.165314 6155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:58.165212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.188:44353 (local address 127.0.0.1:45458)
0807 08:51:58.165264 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:58.165290 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.188:44353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260807 08:51:58.165418 6152 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.159.188:44353: Network error: Client connection negotiation failed: client connection to 127.4.159.188:44353: connect: Connection refused (error 111)
I20260807 08:51:58.165652 6164 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.188:44353
I20260807 08:51:58.165678 6262 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.188:44353 every 8 connection(s)
I20260807 08:51:58.165969 6164 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786092453486433-4734-0/minicluster-data/master-2/data/info.pb
I20260807 08:51:58.166889 6263 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:51:58.167165 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 6164
I20260807 08:51:58.167260 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786092453486433-4734-0/minicluster-data/master-2/wal/instance
I20260807 08:51:58.167433 4734 external_mini_cluster.cc:1428] Setting key 56f8961219609c6dadca6734751a400c
I20260807 08:51:58.168424 6263 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.159.190" port: 36953 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 08:51:58 dist-test-slave-6zbq krb5kdc[5929](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092718, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.188@KRBTEST.COM for kudu/127.4.159.190@KRBTEST.COM
Aug 07 08:51:58 dist-test-slave-6zbq krb5kdc[5929](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092717, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.159.188@KRBTEST.COM
I20260807 08:51:58.175935 6050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:58.169001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44208 (local address 127.4.159.190:36953)
0807 08:51:58.169041 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:58.169043 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:58.169206 (+ 163us) server_negotiation.cc:407] Connection header received
0807 08:51:58.169269 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:58.169270 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:58.169281 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:58.169329 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:51:58.169770 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:58.170235 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:58.171075 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:58.171256 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:58.171312 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:58.173231 (+ 1919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:51:58.173232 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:51:58.173233 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:51:58.173240 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:51:58.174660 (+ 1420us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:51:58.175090 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:51:58.175091 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:51:58.175092 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:51:58.175154 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:51:58.175366 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:51:58.175367 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:51:58.175367 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:51:58.175477 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:51:58.175532 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:58.175773 (+ 241us) server_negotiation.cc:299] Negotiation successful
0807 08:51:58.175851 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 08:51:58.176065 6270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:58.168965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:36953 (local address 127.0.0.1:44208)
0807 08:51:58.169120 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:58.169149 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0807 08:51:58.169211 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:51:58.169358 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:51:58.169361 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:51:58.169587 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:51:58.169678 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:58.169682 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:58.170315 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:58.170316 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:58.170996 (+ 680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:58.170998 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:58.171341 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:58.171342 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:58.171414 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:58.171894 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:51:58.171902 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:51:58.173147 (+ 1245us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:51:58.174732 (+ 1585us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:51:58.174739 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:51:58.174740 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:51:58.175022 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:51:58.175204 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:51:58.175204 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:51:58.175205 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:51:58.175308 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:51:58.175532 (+ 224us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:51:58.175538 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:51:58.175651 (+ 113us) client_negotiation.cc:769] Sending connection context
0807 08:51:58.175736 (+ 85us) client_negotiation.cc:240] Negotiation successful
0807 08:51:58.175868 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":88,"threads_started":1}
I20260807 08:51:58.176142 6269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:58.167880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45466 (local address 127.4.159.188:44353)
0807 08:51:58.168051 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:58.168060 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:58.168085 (+ 25us) server_negotiation.cc:407] Connection header received
0807 08:51:58.168457 (+ 372us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:58.168463 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:58.168506 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:58.168623 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:51:58.169354 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:58.170175 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:58.170926 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:58.171227 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:58.171367 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:58.173628 (+ 2261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:51:58.173636 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:51:58.173636 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:51:58.173643 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:51:58.175101 (+ 1458us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:51:58.175397 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:51:58.175400 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:51:58.175401 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:51:58.175453 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:51:58.175614 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:51:58.175615 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:51:58.175616 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:51:58.175739 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:51:58.175859 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:58.175944 (+ 85us) server_negotiation.cc:299] Negotiation successful
0807 08:51:58.176016 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":76,"threads_started":1}
I20260807 08:51:58.176616 6263 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.159.189" port: 40563 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 08:51:58 dist-test-slave-6zbq krb5kdc[5929](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092718, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.188@KRBTEST.COM for kudu/127.4.159.189@KRBTEST.COM
Aug 07 08:51:58 dist-test-slave-6zbq krb5kdc[5929](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092717, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.159.190@KRBTEST.COM
I20260807 08:51:58.182817 6270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:58.176823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.189:40563 (local address 127.0.0.1:38950)
0807 08:51:58.176919 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:58.176928 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 08:51:58.176967 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:51:58.177075 (+ 108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:51:58.177075 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:51:58.177202 (+ 127us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:51:58.177271 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:58.177272 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:58.178058 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:58.178059 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:58.178777 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:58.178779 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:58.179127 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:58.179128 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:58.179175 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:58.179486 (+ 311us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:51:58.179490 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:51:58.180587 (+ 1097us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:51:58.181949 (+ 1362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:51:58.181950 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:51:58.181951 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:51:58.182108 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:51:58.182275 (+ 167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:51:58.182275 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:51:58.182276 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:51:58.182330 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:51:58.182589 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:51:58.182590 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:51:58.182624 (+ 34us) client_negotiation.cc:769] Sending connection context
0807 08:51:58.182676 (+ 52us) client_negotiation.cc:240] Negotiation successful
0807 08:51:58.182738 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":81}
I20260807 08:51:58.182849 6159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:58.176878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38950 (local address 127.4.159.189:40563)
0807 08:51:58.176922 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:58.176923 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:58.176958 (+ 35us) server_negotiation.cc:407] Connection header received
0807 08:51:58.177014 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:58.177015 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:58.177024 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:58.177070 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:51:58.177336 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:58.177982 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:58.178854 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:58.179059 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:58.179124 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:58.180672 (+ 1548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:51:58.180674 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:51:58.180674 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:51:58.180681 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:51:58.181876 (+ 1195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:51:58.182168 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:51:58.182169 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:51:58.182169 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:51:58.182215 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:51:58.182410 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:51:58.182411 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:51:58.182411 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:51:58.182520 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:51:58.182585 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:58.182715 (+ 130us) server_negotiation.cc:299] Negotiation successful
0807 08:51:58.182792 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 08:51:58.183141 6263 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.159.188" port: 44353 } has no permanent_uuid. Determining permanent_uuid...
I20260807 08:51:58.183589 6050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:58.176843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44210 (local address 127.4.159.190:36953)
0807 08:51:58.176875 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:58.176877 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:58.176884 (+ 7us) server_negotiation.cc:407] Connection header received
0807 08:51:58.176937 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:58.176938 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:58.176946 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:58.176997 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:51:58.177487 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:58.178139 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:58.179007 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:58.179234 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:58.179289 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:58.181202 (+ 1913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:51:58.181203 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:51:58.181204 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:51:58.181210 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:51:58.182454 (+ 1244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:51:58.182786 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:51:58.182787 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:51:58.182788 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:51:58.182842 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:51:58.183041 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:51:58.183042 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:51:58.183042 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:51:58.183231 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:51:58.183390 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:58.183431 (+ 41us) server_negotiation.cc:299] Negotiation successful
0807 08:51:58.183510 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
Aug 07 08:51:58 dist-test-slave-6zbq krb5kdc[5929](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092718, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.188@KRBTEST.COM for kudu/127.4.159.188@KRBTEST.COM
Aug 07 08:51:58 dist-test-slave-6zbq krb5kdc[5929](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092717, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.159.189@KRBTEST.COM
I20260807 08:51:58.189206 6270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:58.183339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.188:44353 (local address 127.0.0.1:45470)
0807 08:51:58.183381 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:58.183441 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0807 08:51:58.183483 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:51:58.183589 (+ 106us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:51:58.183589 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:51:58.183712 (+ 123us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:51:58.183739 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:58.183741 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:58.184507 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:58.184508 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:58.185164 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:58.185166 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:58.185518 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:58.185519 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:58.185555 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:58.185861 (+ 306us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:51:58.185866 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:51:58.187081 (+ 1215us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:51:58.188380 (+ 1299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:51:58.188381 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:51:58.188381 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:51:58.188519 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:51:58.188672 (+ 153us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:51:58.188673 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:51:58.188673 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:51:58.188721 (+ 48us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:51:58.188914 (+ 193us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:51:58.188915 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:51:58.188948 (+ 33us) client_negotiation.cc:769] Sending connection context
0807 08:51:58.189005 (+ 57us) client_negotiation.cc:240] Negotiation successful
0807 08:51:58.189095 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260807 08:51:58.189265 6269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:58.183413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45470 (local address 127.4.159.188:44353)
0807 08:51:58.183447 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:58.183449 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:58.183470 (+ 21us) server_negotiation.cc:407] Connection header received
0807 08:51:58.183525 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:58.183526 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:58.183534 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:58.183584 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:51:58.183806 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:58.184438 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:58.185237 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:58.185452 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:58.185514 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:58.187151 (+ 1637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:51:58.187152 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:51:58.187153 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:51:58.187158 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:51:58.188307 (+ 1149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:51:58.188577 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:51:58.188578 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:51:58.188578 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:51:58.188623 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:51:58.188773 (+ 150us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:51:58.188774 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:51:58.188774 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:51:58.188862 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:51:58.188912 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:58.189030 (+ 118us) server_negotiation.cc:299] Negotiation successful
0807 08:51:58.189143 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":57,"server-negotiator.queue_time_us":17}
I20260807 08:51:58.190615 6159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:58.184125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38966 (local address 127.4.159.189:40563)
0807 08:51:58.184157 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:58.184158 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:58.184167 (+ 9us) server_negotiation.cc:407] Connection header received
0807 08:51:58.184187 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:58.184187 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:58.184195 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:58.184238 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:51:58.184708 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:58.185342 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:58.185962 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:58.186239 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:58.186317 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:58.188203 (+ 1886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:51:58.188204 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:51:58.188204 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:51:58.188210 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:51:58.189572 (+ 1362us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:51:58.189914 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:51:58.189915 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:51:58.189915 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:51:58.189974 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:51:58.190134 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:51:58.190135 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:51:58.190135 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:51:58.190253 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:51:58.190426 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:58.190467 (+ 41us) server_negotiation.cc:299] Negotiation successful
0807 08:51:58.190536 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260807 08:51:58.190812 6263 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b85be12e84fa44ad98a556ee5c488a80: Bootstrap starting.
I20260807 08:51:58.191715 6263 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b85be12e84fa44ad98a556ee5c488a80: Neither blocks nor log segments found. Creating new log.
I20260807 08:51:58.191913 4734 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 08:51:58.192016 6263 log.cc:824] T 00000000000000000000000000000000 P b85be12e84fa44ad98a556ee5c488a80: Log is configured to *not* fsync() on all Append() calls
I20260807 08:51:58.192772 6263 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b85be12e84fa44ad98a556ee5c488a80: No bootstrap required, opened a new log
I20260807 08:51:58.192921 6008 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "67b1b2da5df841f89e9c42793cb37b1a"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44210
I20260807 08:51:58.194165 6263 raft_consensus.cc:348] T 00000000000000000000000000000000 P b85be12e84fa44ad98a556ee5c488a80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67b1b2da5df841f89e9c42793cb37b1a" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 36953 } } peers { permanent_uuid: "e64815bd02294e9c9dc880fc2fb5fbdf" member_type: VOTER last_known_addr { host: "127.4.159.189" port: 40563 } } peers { permanent_uuid: "b85be12e84fa44ad98a556ee5c488a80" member_type: VOTER last_known_addr { host: "127.4.159.188" port: 44353 } }
I20260807 08:51:58.194280 6263 raft_consensus.cc:729] T 00000000000000000000000000000000 P b85be12e84fa44ad98a556ee5c488a80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b85be12e84fa44ad98a556ee5c488a80, State: Initialized, Role: FOLLOWER
I20260807 08:51:58.194380 6263 consensus_queue.cc:260] T 00000000000000000000000000000000 P b85be12e84fa44ad98a556ee5c488a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67b1b2da5df841f89e9c42793cb37b1a" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 36953 } } peers { permanent_uuid: "e64815bd02294e9c9dc880fc2fb5fbdf" member_type: VOTER last_known_addr { host: "127.4.159.189" port: 40563 } } peers { permanent_uuid: "b85be12e84fa44ad98a556ee5c488a80" member_type: VOTER last_known_addr { host: "127.4.159.188" port: 44353 } }
I20260807 08:51:58.194672 6263 sys_catalog.cc:564] T 00000000000000000000000000000000 P b85be12e84fa44ad98a556ee5c488a80 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:51:58.194635 6272 sys_catalog.cc:455] T 00000000000000000000000000000000 P b85be12e84fa44ad98a556ee5c488a80 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67b1b2da5df841f89e9c42793cb37b1a" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 36953 } } peers { permanent_uuid: "e64815bd02294e9c9dc880fc2fb5fbdf" member_type: VOTER last_known_addr { host: "127.4.159.189" port: 40563 } } peers { permanent_uuid: "b85be12e84fa44ad98a556ee5c488a80" member_type: VOTER last_known_addr { host: "127.4.159.188" port: 44353 } } }
I20260807 08:51:58.194722 6272 sys_catalog.cc:458] T 00000000000000000000000000000000 P b85be12e84fa44ad98a556ee5c488a80 [sys.catalog]: This master's current role is: FOLLOWER
W20260807 08:51:58.196177 6284 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b85be12e84fa44ad98a556ee5c488a80: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 08:51:58.196233 6284 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 08:51:58.196403 6164 master_runner.cc:186] Error getting master registration for 127.4.159.190:36953: OK, instance_id { permanent_uuid: "67b1b2da5df841f89e9c42793cb37b1a" instance_seqno: 1786092717863418 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 08:51:58.199076 6164 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 08:51:58.203171 6045 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.159.188" port: 44353 } attempt: 2
Aug 07 08:51:58 dist-test-slave-6zbq krb5kdc[5929](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092717, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.190@KRBTEST.COM for kudu/127.4.159.188@KRBTEST.COM
I20260807 08:51:58.210610 6269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:58.203464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45474 (local address 127.4.159.188:44353)
0807 08:51:58.203502 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:58.203504 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:58.203512 (+ 8us) server_negotiation.cc:407] Connection header received
0807 08:51:58.203531 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:58.203532 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:58.203540 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:58.203606 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:51:58.203923 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:58.204625 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:58.205582 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:58.205882 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:58.205947 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:58.207775 (+ 1828us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:51:58.207777 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:51:58.207777 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:51:58.207783 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:51:58.209071 (+ 1288us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:51:58.209681 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:51:58.209683 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:51:58.209683 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:51:58.209753 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:51:58.209955 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:51:58.209955 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:51:58.209956 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:51:58.210188 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:51:58.210270 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:58.210444 (+ 174us) server_negotiation.cc:299] Negotiation successful
0807 08:51:58.210521 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 08:51:58.210611 6049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:58.203391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.188:44353 (local address 127.0.0.1:45474)
0807 08:51:58.203439 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:58.203448 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 08:51:58.203484 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:51:58.203638 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:51:58.203639 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:51:58.203806 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:51:58.203849 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:58.203851 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:58.204701 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:58.204702 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:58.205512 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:58.205514 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:58.205948 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:58.205948 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:58.205998 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:58.206393 (+ 395us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:51:58.206398 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:51:58.207692 (+ 1294us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:51:58.209234 (+ 1542us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:51:58.209235 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:51:58.209236 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:51:58.209449 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:51:58.209831 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:51:58.209831 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:51:58.209832 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:51:58.209897 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:51:58.210264 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:51:58.210265 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:51:58.210313 (+ 48us) client_negotiation.cc:769] Sending connection context
0807 08:51:58.210418 (+ 105us) client_negotiation.cc:240] Negotiation successful
0807 08:51:58.210513 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260807 08:51:58.212174 6045 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 67b1b2da5df841f89e9c42793cb37b1a: Bootstrap starting.
I20260807 08:51:58.212774 6045 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 67b1b2da5df841f89e9c42793cb37b1a: Neither blocks nor log segments found. Creating new log.
I20260807 08:51:58.213020 6045 log.cc:824] T 00000000000000000000000000000000 P 67b1b2da5df841f89e9c42793cb37b1a: Log is configured to *not* fsync() on all Append() calls
I20260807 08:51:58.213682 6045 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 67b1b2da5df841f89e9c42793cb37b1a: No bootstrap required, opened a new log
I20260807 08:51:58.214993 6045 raft_consensus.cc:348] T 00000000000000000000000000000000 P 67b1b2da5df841f89e9c42793cb37b1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67b1b2da5df841f89e9c42793cb37b1a" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 36953 } } peers { permanent_uuid: "e64815bd02294e9c9dc880fc2fb5fbdf" member_type: VOTER last_known_addr { host: "127.4.159.189" port: 40563 } } peers { permanent_uuid: "b85be12e84fa44ad98a556ee5c488a80" member_type: VOTER last_known_addr { host: "127.4.159.188" port: 44353 } }
I20260807 08:51:58.215126 6045 raft_consensus.cc:729] T 00000000000000000000000000000000 P 67b1b2da5df841f89e9c42793cb37b1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67b1b2da5df841f89e9c42793cb37b1a, State: Initialized, Role: FOLLOWER
I20260807 08:51:58.215201 6045 consensus_queue.cc:260] T 00000000000000000000000000000000 P 67b1b2da5df841f89e9c42793cb37b1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67b1b2da5df841f89e9c42793cb37b1a" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 36953 } } peers { permanent_uuid: "e64815bd02294e9c9dc880fc2fb5fbdf" member_type: VOTER last_known_addr { host: "127.4.159.189" port: 40563 } } peers { permanent_uuid: "b85be12e84fa44ad98a556ee5c488a80" member_type: VOTER last_known_addr { host: "127.4.159.188" port: 44353 } }
I20260807 08:51:58.215443 6288 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67b1b2da5df841f89e9c42793cb37b1a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67b1b2da5df841f89e9c42793cb37b1a" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 36953 } } peers { permanent_uuid: "e64815bd02294e9c9dc880fc2fb5fbdf" member_type: VOTER last_known_addr { host: "127.4.159.189" port: 40563 } } peers { permanent_uuid: "b85be12e84fa44ad98a556ee5c488a80" member_type: VOTER last_known_addr { host: "127.4.159.188" port: 44353 } } }
I20260807 08:51:58.215497 6045 sys_catalog.cc:564] T 00000000000000000000000000000000 P 67b1b2da5df841f89e9c42793cb37b1a [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:51:58.215520 6288 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67b1b2da5df841f89e9c42793cb37b1a [sys.catalog]: This master's current role is: FOLLOWER
I20260807 08:51:58.215637 6008 raft_consensus.cc:482] T 00000000000000000000000000000000 P 67b1b2da5df841f89e9c42793cb37b1a [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 08:51:58.215718 6008 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 67b1b2da5df841f89e9c42793cb37b1a [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:51:58.216466 6008 raft_consensus.cc:504] T 00000000000000000000000000000000 P 67b1b2da5df841f89e9c42793cb37b1a [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67b1b2da5df841f89e9c42793cb37b1a" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 36953 } } peers { permanent_uuid: "e64815bd02294e9c9dc880fc2fb5fbdf" member_type: VOTER last_known_addr { host: "127.4.159.189" port: 40563 } } peers { permanent_uuid: "b85be12e84fa44ad98a556ee5c488a80" member_type: VOTER last_known_addr { host: "127.4.159.188" port: 44353 } }
I20260807 08:51:58.217325 6223 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "67b1b2da5df841f89e9c42793cb37b1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "b85be12e84fa44ad98a556ee5c488a80"
I20260807 08:51:58.217447 6223 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b85be12e84fa44ad98a556ee5c488a80 [term 0 FOLLOWER]: Advancing to term 1
W20260807 08:51:58.217773 6301 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 67b1b2da5df841f89e9c42793cb37b1a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 08:51:58.217887 6301 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 08:51:58.218286 6223 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b85be12e84fa44ad98a556ee5c488a80 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67b1b2da5df841f89e9c42793cb37b1a in term 1.
I20260807 08:51:58.218566 5977 leader_election.cc:302] T 00000000000000000000000000000000 P 67b1b2da5df841f89e9c42793cb37b1a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 67b1b2da5df841f89e9c42793cb37b1a, b85be12e84fa44ad98a556ee5c488a80; no voters:
I20260807 08:51:58.218689 6288 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 67b1b2da5df841f89e9c42793cb37b1a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:51:58.218818 6288 raft_consensus.cc:686] T 00000000000000000000000000000000 P 67b1b2da5df841f89e9c42793cb37b1a [term 1 LEADER]: Becoming Leader. State: Replica: 67b1b2da5df841f89e9c42793cb37b1a, State: Running, Role: LEADER
I20260807 08:51:58.218902 6288 consensus_queue.cc:237] T 00000000000000000000000000000000 P 67b1b2da5df841f89e9c42793cb37b1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67b1b2da5df841f89e9c42793cb37b1a" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 36953 } } peers { permanent_uuid: "e64815bd02294e9c9dc880fc2fb5fbdf" member_type: VOTER last_known_addr { host: "127.4.159.189" port: 40563 } } peers { permanent_uuid: "b85be12e84fa44ad98a556ee5c488a80" member_type: VOTER last_known_addr { host: "127.4.159.188" port: 44353 } }
I20260807 08:51:58.219280 6288 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67b1b2da5df841f89e9c42793cb37b1a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 67b1b2da5df841f89e9c42793cb37b1a. Latest consensus state: current_term: 1 leader_uuid: "67b1b2da5df841f89e9c42793cb37b1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67b1b2da5df841f89e9c42793cb37b1a" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 36953 } } peers { permanent_uuid: "e64815bd02294e9c9dc880fc2fb5fbdf" member_type: VOTER last_known_addr { host: "127.4.159.189" port: 40563 } } peers { permanent_uuid: "b85be12e84fa44ad98a556ee5c488a80" member_type: VOTER last_known_addr { host: "127.4.159.188" port: 44353 } } }
I20260807 08:51:58.219355 6288 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67b1b2da5df841f89e9c42793cb37b1a [sys.catalog]: This master's current role is: LEADER
I20260807 08:51:58.219504 6008 leader_election.cc:288] T 00000000000000000000000000000000 P 67b1b2da5df841f89e9c42793cb37b1a [CANDIDATE]: Term 1 election: Requested vote from peers e64815bd02294e9c9dc880fc2fb5fbdf (127.4.159.189:40563), b85be12e84fa44ad98a556ee5c488a80 (127.4.159.188:44353)
I20260807 08:51:58.218498 5946 master_runner.cc:186] Error getting master registration for 127.4.159.189:40563: OK, instance_id { permanent_uuid: "e64815bd02294e9c9dc880fc2fb5fbdf" instance_seqno: 1786092718024347 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 08:51:58.220672 5946 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 08:51:58.220577 6307 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:51:58.220876 6307 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:51:58.222648 6223 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b85be12e84fa44ad98a556ee5c488a80 [term 1 FOLLOWER]: Refusing update from remote peer 67b1b2da5df841f89e9c42793cb37b1a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:51:58.222959 6305 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 67b1b2da5df841f89e9c42793cb37b1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "b85be12e84fa44ad98a556ee5c488a80" member_type: VOTER last_known_addr { host: "127.4.159.188" port: 44353 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:51:58.224162 6272 sys_catalog.cc:455] T 00000000000000000000000000000000 P b85be12e84fa44ad98a556ee5c488a80 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 67b1b2da5df841f89e9c42793cb37b1a. Latest consensus state: current_term: 1 leader_uuid: "67b1b2da5df841f89e9c42793cb37b1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67b1b2da5df841f89e9c42793cb37b1a" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 36953 } } peers { permanent_uuid: "e64815bd02294e9c9dc880fc2fb5fbdf" member_type: VOTER last_known_addr { host: "127.4.159.189" port: 40563 } } peers { permanent_uuid: "b85be12e84fa44ad98a556ee5c488a80" member_type: VOTER last_known_addr { host: "127.4.159.188" port: 44353 } } }
I20260807 08:51:58.224254 6272 sys_catalog.cc:458] T 00000000000000000000000000000000 P b85be12e84fa44ad98a556ee5c488a80 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 08:51:58.234370 6050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:58.221332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44224 (local address 127.4.159.190:36953)
0807 08:51:58.221367 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:58.221369 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:58.226608 (+ 5239us) server_negotiation.cc:407] Connection header received
0807 08:51:58.226657 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:58.226658 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:58.226667 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:58.226710 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:51:58.227409 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:58.228125 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:58.229161 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:58.229451 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:58.229490 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:58.230984 (+ 1494us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:51:58.230985 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:51:58.230985 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:51:58.230992 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:51:58.232687 (+ 1695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:51:58.233313 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:51:58.233314 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:51:58.233315 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:51:58.233371 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:51:58.233788 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:51:58.233789 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:51:58.233789 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:51:58.233908 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:51:58.233986 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:58.234176 (+ 190us) server_negotiation.cc:299] Negotiation successful
0807 08:51:58.234246 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 08:51:58.235491 6314 mvcc.cc:205] Tried to move back new op lower bound from 7315835773836050432 to 7315835773825302528. Current Snapshot: MvccSnapshot[applied={T|T < 7315835773836050432}]
I20260807 08:51:58.235721 6288 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67b1b2da5df841f89e9c42793cb37b1a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "67b1b2da5df841f89e9c42793cb37b1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67b1b2da5df841f89e9c42793cb37b1a" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 36953 } } peers { permanent_uuid: "e64815bd02294e9c9dc880fc2fb5fbdf" member_type: VOTER last_known_addr { host: "127.4.159.189" port: 40563 } } peers { permanent_uuid: "b85be12e84fa44ad98a556ee5c488a80" member_type: VOTER last_known_addr { host: "127.4.159.188" port: 44353 } } }
I20260807 08:51:58.235809 6288 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67b1b2da5df841f89e9c42793cb37b1a [sys.catalog]: This master's current role is: LEADER
I20260807 08:51:58.236236 6272 sys_catalog.cc:455] T 00000000000000000000000000000000 P b85be12e84fa44ad98a556ee5c488a80 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "67b1b2da5df841f89e9c42793cb37b1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67b1b2da5df841f89e9c42793cb37b1a" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 36953 } } peers { permanent_uuid: "e64815bd02294e9c9dc880fc2fb5fbdf" member_type: VOTER last_known_addr { host: "127.4.159.189" port: 40563 } } peers { permanent_uuid: "b85be12e84fa44ad98a556ee5c488a80" member_type: VOTER last_known_addr { host: "127.4.159.188" port: 44353 } } }
I20260807 08:51:58.236311 6272 sys_catalog.cc:458] T 00000000000000000000000000000000 P b85be12e84fa44ad98a556ee5c488a80 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 08:51:58.236616 6269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:58.221138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45476 (local address 127.4.159.188:44353)
0807 08:51:58.221171 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:58.221172 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:58.226465 (+ 5293us) server_negotiation.cc:407] Connection header received
0807 08:51:58.226546 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:58.226548 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:58.226559 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:58.226620 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:51:58.227892 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:58.228591 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:58.230290 (+ 1699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:58.230609 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:58.230647 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:58.232739 (+ 2092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:51:58.232741 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:51:58.232741 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:51:58.232747 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:51:58.234228 (+ 1481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:51:58.235404 (+ 1176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:51:58.235420 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:51:58.235420 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:51:58.235478 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:51:58.235851 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:51:58.235852 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:51:58.235852 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:51:58.235990 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:51:58.236055 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:58.236444 (+ 389us) server_negotiation.cc:299] Negotiation successful
0807 08:51:58.236518 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260807 08:51:58.236517 6320 mvcc.cc:205] Tried to move back new op lower bound from 7315835773836050432 to 7315835773825302528. Current Snapshot: MvccSnapshot[applied={T|T < 7315835773836050432}]
I20260807 08:51:58.242197 6159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:58.222036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38972 (local address 127.4.159.189:40563)
0807 08:51:58.222146 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:58.222148 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:58.234636 (+ 12488us) server_negotiation.cc:407] Connection header received
0807 08:51:58.234735 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:58.234736 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:58.234746 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:58.234773 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:51:58.236137 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:58.236790 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:58.237790 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:58.238116 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:58.238185 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:58.239487 (+ 1302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:51:58.239488 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:51:58.239489 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:51:58.239495 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:51:58.241040 (+ 1545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:51:58.241430 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:51:58.241430 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:51:58.241431 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:51:58.241502 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:51:58.241667 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:51:58.241668 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:51:58.241668 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:51:58.241810 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:51:58.241894 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:58.242019 (+ 125us) server_negotiation.cc:299] Negotiation successful
0807 08:51:58.242105 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260807 08:51:58.242998 6307 catalog_manager.cc:1347] Generated new cluster ID: 9695023d50784c7cbb58d27dd91989f1
I20260807 08:51:58.243050 6307 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:51:58.256543 6152 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.159.188" port: 44353 } attempt: 3
I20260807 08:51:58.260035 6307 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:51:58.260812 6307 catalog_manager.cc:1504] Loading token signing keys...
Aug 07 08:51:58 dist-test-slave-6zbq krb5kdc[5929](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092717, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.189@KRBTEST.COM for kudu/127.4.159.188@KRBTEST.COM
I20260807 08:51:58.265240 6155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:58.256806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.188:44353 (local address 127.0.0.1:45490)
0807 08:51:58.256875 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0807 08:51:58.256883 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0807 08:51:58.256915 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:51:58.257109 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:51:58.257110 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:51:58.257255 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:51:58.257302 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:58.257303 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:58.258165 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:58.258166 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:58.258970 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:51:58.258973 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:51:58.260187 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:51:58.260188 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:51:58.260262 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:58.260703 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:51:58.260708 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:51:58.262148 (+ 1440us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:51:58.264120 (+ 1972us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:51:58.264121 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:51:58.264121 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:51:58.264341 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:51:58.264578 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:51:58.264579 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:51:58.264579 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:51:58.264647 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:51:58.264942 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:51:58.264943 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:51:58.264998 (+ 55us) client_negotiation.cc:769] Sending connection context
0807 08:51:58.265055 (+ 57us) client_negotiation.cc:240] Negotiation successful
0807 08:51:58.265152 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260807 08:51:58.266400 6269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:58.256974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45490 (local address 127.4.159.188:44353)
0807 08:51:58.257005 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:58.257007 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:58.257012 (+ 5us) server_negotiation.cc:407] Connection header received
0807 08:51:58.257029 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:58.257030 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:58.257037 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:58.257077 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:51:58.257376 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:58.258079 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:58.259733 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:58.260019 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:58.260075 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:58.262248 (+ 2173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:51:58.262249 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:51:58.262250 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:51:58.262255 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:51:58.264004 (+ 1749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:51:58.264430 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:51:58.264431 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:51:58.264432 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:51:58.264498 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:51:58.264719 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:51:58.264720 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:51:58.264720 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:51:58.264847 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:51:58.264901 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:58.266223 (+ 1322us) server_negotiation.cc:299] Negotiation successful
0807 08:51:58.266302 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260807 08:51:58.268006 6152 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e64815bd02294e9c9dc880fc2fb5fbdf: Bootstrap starting.
I20260807 08:51:58.268152 6307 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 67b1b2da5df841f89e9c42793cb37b1a: Generated new TSK 0
I20260807 08:51:58.268303 6307 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:51:58.268700 6152 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e64815bd02294e9c9dc880fc2fb5fbdf: Neither blocks nor log segments found. Creating new log.
I20260807 08:51:58.268970 6152 log.cc:824] T 00000000000000000000000000000000 P e64815bd02294e9c9dc880fc2fb5fbdf: Log is configured to *not* fsync() on all Append() calls
I20260807 08:51:58.269798 6152 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e64815bd02294e9c9dc880fc2fb5fbdf: No bootstrap required, opened a new log
I20260807 08:51:58.273123 6152 raft_consensus.cc:348] T 00000000000000000000000000000000 P e64815bd02294e9c9dc880fc2fb5fbdf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67b1b2da5df841f89e9c42793cb37b1a" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 36953 } } peers { permanent_uuid: "e64815bd02294e9c9dc880fc2fb5fbdf" member_type: VOTER last_known_addr { host: "127.4.159.189" port: 40563 } } peers { permanent_uuid: "b85be12e84fa44ad98a556ee5c488a80" member_type: VOTER last_known_addr { host: "127.4.159.188" port: 44353 } }
I20260807 08:51:58.273257 6152 raft_consensus.cc:729] T 00000000000000000000000000000000 P e64815bd02294e9c9dc880fc2fb5fbdf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e64815bd02294e9c9dc880fc2fb5fbdf, State: Initialized, Role: FOLLOWER
I20260807 08:51:58.273344 6152 consensus_queue.cc:260] T 00000000000000000000000000000000 P e64815bd02294e9c9dc880fc2fb5fbdf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67b1b2da5df841f89e9c42793cb37b1a" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 36953 } } peers { permanent_uuid: "e64815bd02294e9c9dc880fc2fb5fbdf" member_type: VOTER last_known_addr { host: "127.4.159.189" port: 40563 } } peers { permanent_uuid: "b85be12e84fa44ad98a556ee5c488a80" member_type: VOTER last_known_addr { host: "127.4.159.188" port: 44353 } }
I20260807 08:51:58.273739 6112 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e64815bd02294e9c9dc880fc2fb5fbdf [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:51:58.274623 6112 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e64815bd02294e9c9dc880fc2fb5fbdf [term 1 FOLLOWER]: Refusing update from remote peer 67b1b2da5df841f89e9c42793cb37b1a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:51:58.274806 6113 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "67b1b2da5df841f89e9c42793cb37b1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "e64815bd02294e9c9dc880fc2fb5fbdf"
I20260807 08:51:58.275497 6113 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e64815bd02294e9c9dc880fc2fb5fbdf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67b1b2da5df841f89e9c42793cb37b1a in term 1.
I20260807 08:51:58.275841 6323 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 67b1b2da5df841f89e9c42793cb37b1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64815bd02294e9c9dc880fc2fb5fbdf" member_type: VOTER last_known_addr { host: "127.4.159.189" port: 40563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:51:58.273649 6152 sys_catalog.cc:564] T 00000000000000000000000000000000 P e64815bd02294e9c9dc880fc2fb5fbdf [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:51:58.273650 6326 sys_catalog.cc:455] T 00000000000000000000000000000000 P e64815bd02294e9c9dc880fc2fb5fbdf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67b1b2da5df841f89e9c42793cb37b1a" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 36953 } } peers { permanent_uuid: "e64815bd02294e9c9dc880fc2fb5fbdf" member_type: VOTER last_known_addr { host: "127.4.159.189" port: 40563 } } peers { permanent_uuid: "b85be12e84fa44ad98a556ee5c488a80" member_type: VOTER last_known_addr { host: "127.4.159.188" port: 44353 } } }
I20260807 08:51:58.279003 6326 sys_catalog.cc:458] T 00000000000000000000000000000000 P e64815bd02294e9c9dc880fc2fb5fbdf [sys.catalog]: This master's current role is: FOLLOWER
I20260807 08:51:58.279119 6326 sys_catalog.cc:455] T 00000000000000000000000000000000 P e64815bd02294e9c9dc880fc2fb5fbdf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 67b1b2da5df841f89e9c42793cb37b1a. Latest consensus state: current_term: 1 leader_uuid: "67b1b2da5df841f89e9c42793cb37b1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67b1b2da5df841f89e9c42793cb37b1a" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 36953 } } peers { permanent_uuid: "e64815bd02294e9c9dc880fc2fb5fbdf" member_type: VOTER last_known_addr { host: "127.4.159.189" port: 40563 } } peers { permanent_uuid: "b85be12e84fa44ad98a556ee5c488a80" member_type: VOTER last_known_addr { host: "127.4.159.188" port: 44353 } } }
I20260807 08:51:58.279240 6326 sys_catalog.cc:458] T 00000000000000000000000000000000 P e64815bd02294e9c9dc880fc2fb5fbdf [sys.catalog]: This master's current role is: FOLLOWER
I20260807 08:51:58.279321 6326 sys_catalog.cc:455] T 00000000000000000000000000000000 P e64815bd02294e9c9dc880fc2fb5fbdf [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "67b1b2da5df841f89e9c42793cb37b1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67b1b2da5df841f89e9c42793cb37b1a" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 36953 } } peers { permanent_uuid: "e64815bd02294e9c9dc880fc2fb5fbdf" member_type: VOTER last_known_addr { host: "127.4.159.189" port: 40563 } } peers { permanent_uuid: "b85be12e84fa44ad98a556ee5c488a80" member_type: VOTER last_known_addr { host: "127.4.159.188" port: 44353 } } }
I20260807 08:51:58.279426 6326 sys_catalog.cc:458] T 00000000000000000000000000000000 P e64815bd02294e9c9dc880fc2fb5fbdf [sys.catalog]: This master's current role is: FOLLOWER
I20260807 08:51:58.283484 6330 mvcc.cc:205] Tried to move back new op lower bound from 7315835774021562368 to 7315835773825302528. Current Snapshot: MvccSnapshot[applied={T|T < 7315835773836050432}]
W20260807 08:51:58.284224 6353 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e64815bd02294e9c9dc880fc2fb5fbdf: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 08:51:58.284322 6353 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260807 08:51:58.290911 6269 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:51:58.289684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45502 (local address 127.4.159.188:44353)
0807 08:51:58.289719 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:58.289721 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:58.290309 (+ 588us) server_negotiation.cc:407] Connection header received
0807 08:51:58.290423 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:58.290424 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:58.290434 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:58.290461 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:51:58.290857 (+ 396us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45502: BlockingRecv error: recv got EOF from 127.0.0.1:45502 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260807 08:51:58.291584 6050 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:51:58.290050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44238 (local address 127.4.159.190:36953)
0807 08:51:58.290097 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:58.290099 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:58.291145 (+ 1046us) server_negotiation.cc:407] Connection header received
0807 08:51:58.291228 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:58.291229 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:58.291238 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:58.291264 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:51:58.291529 (+ 265us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44238: BlockingRecv error: recv got EOF from 127.0.0.1:44238 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260807 08:51:58.292448 6159 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:51:58.291870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38986 (local address 127.4.159.189:40563)
0807 08:51:58.291909 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:58.291911 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:58.292014 (+ 103us) server_negotiation.cc:407] Connection header received
0807 08:51:58.292103 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:58.292104 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:58.292113 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:58.292139 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:51:58.292399 (+ 260us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38986: BlockingRecv error: recv got EOF from 127.0.0.1:38986 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260807 08:51:58.294595 6159 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:51:58.293858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38994 (local address 127.4.159.189:40563)
0807 08:51:58.293889 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:58.293890 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:58.294040 (+ 150us) server_negotiation.cc:407] Connection header received
0807 08:51:58.294145 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:58.294146 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:58.294161 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:58.294185 (+ 24us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:51:58.294557 (+ 372us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38994: BlockingRecv error: recv got EOF from 127.0.0.1:38994 (error 108)
Metrics: {"server-negotiator.queue_time_us":14}
W20260807 08:51:58.297426 6159 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:51:58.296747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39000 (local address 127.4.159.189:40563)
0807 08:51:58.296777 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:58.296779 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:58.296968 (+ 189us) server_negotiation.cc:407] Connection header received
0807 08:51:58.297049 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:58.297050 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:58.297059 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:58.297097 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:51:58.297388 (+ 291us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39000: BlockingRecv error: recv got EOF from 127.0.0.1:39000 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260807 08:51:58.301214 6159 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:51:58.300470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39004 (local address 127.4.159.189:40563)
0807 08:51:58.300516 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:58.300517 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:58.300719 (+ 202us) server_negotiation.cc:407] Connection header received
0807 08:51:58.300811 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:58.300812 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:58.300822 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:58.300848 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:51:58.301176 (+ 328us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39004: BlockingRecv error: recv got EOF from 127.0.0.1:39004 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260807 08:51:58.307075 6159 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:51:58.306296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39006 (local address 127.4.159.189:40563)
0807 08:51:58.306329 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:58.306331 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:58.306604 (+ 273us) server_negotiation.cc:407] Connection header received
0807 08:51:58.306698 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:58.306699 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:58.306708 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:58.306735 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:51:58.307033 (+ 298us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39006: BlockingRecv error: recv got EOF from 127.0.0.1:39006 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260807 08:51:58.316993 6159 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:51:58.316213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39016 (local address 127.4.159.189:40563)
0807 08:51:58.316244 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:58.316246 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:58.316492 (+ 246us) server_negotiation.cc:407] Connection header received
0807 08:51:58.316589 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:58.316590 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:58.316601 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:58.316630 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:51:58.316949 (+ 319us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39016: BlockingRecv error: recv got EOF from 127.0.0.1:39016 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260807 08:51:58.334944 6159 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:51:58.334167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39030 (local address 127.4.159.189:40563)
0807 08:51:58.334204 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:58.334205 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:58.334451 (+ 246us) server_negotiation.cc:407] Connection header received
0807 08:51:58.334543 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:58.334544 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:58.334553 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:58.334580 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:51:58.334896 (+ 316us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39030: BlockingRecv error: recv got EOF from 127.0.0.1:39030 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260807 08:51:58.368721 6159 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:51:58.368092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39034 (local address 127.4.159.189:40563)
0807 08:51:58.368134 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:58.368136 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:58.368243 (+ 107us) server_negotiation.cc:407] Connection header received
0807 08:51:58.368300 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:58.368300 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:58.368312 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:58.368360 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:51:58.368662 (+ 302us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39034: BlockingRecv error: recv got EOF from 127.0.0.1:39034 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260807 08:51:58.434427 6159 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:51:58.433740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39048 (local address 127.4.159.189:40563)
0807 08:51:58.433794 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:58.433796 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:58.433842 (+ 46us) server_negotiation.cc:407] Connection header received
0807 08:51:58.433895 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:58.433896 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:58.433909 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:58.433988 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:51:58.434366 (+ 378us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39048: BlockingRecv error: recv got EOF from 127.0.0.1:39048 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
W20260807 08:51:58.563954 6159 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:51:58.563369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39056 (local address 127.4.159.189:40563)
0807 08:51:58.563442 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:58.563444 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:58.563502 (+ 58us) server_negotiation.cc:407] Connection header received
0807 08:51:58.563566 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:58.563567 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:58.563577 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:58.563614 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:51:58.563896 (+ 282us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39056: BlockingRecv error: recv got EOF from 127.0.0.1:39056 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260807 08:51:58.821581 6159 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:51:58.821013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39064 (local address 127.4.159.189:40563)
0807 08:51:58.821088 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:58.821090 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:58.821105 (+ 15us) server_negotiation.cc:407] Connection header received
0807 08:51:58.821173 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:58.821174 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:58.821185 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:58.821234 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:51:58.821525 (+ 291us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39064: BlockingRecv error: recv got EOF from 127.0.0.1:39064 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
I20260807 08:51:58.831760 6323 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67b1b2da5df841f89e9c42793cb37b1a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "67b1b2da5df841f89e9c42793cb37b1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67b1b2da5df841f89e9c42793cb37b1a" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 36953 } } peers { permanent_uuid: "e64815bd02294e9c9dc880fc2fb5fbdf" member_type: VOTER last_known_addr { host: "127.4.159.189" port: 40563 } } peers { permanent_uuid: "b85be12e84fa44ad98a556ee5c488a80" member_type: VOTER last_known_addr { host: "127.4.159.188" port: 44353 } } }
I20260807 08:51:58.831894 6323 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67b1b2da5df841f89e9c42793cb37b1a [sys.catalog]: This master's current role is: LEADER
I20260807 08:51:59.196645 6284 catalog_manager.cc:1259] Loaded cluster ID: 9695023d50784c7cbb58d27dd91989f1
I20260807 08:51:59.196707 6284 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b85be12e84fa44ad98a556ee5c488a80: loading cluster ID for follower catalog manager: success
I20260807 08:51:59.197429 6284 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b85be12e84fa44ad98a556ee5c488a80: acquiring CA information for follower catalog manager: success
I20260807 08:51:59.197681 6284 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b85be12e84fa44ad98a556ee5c488a80: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 08:51:59.284766 6353 catalog_manager.cc:1259] Loaded cluster ID: 9695023d50784c7cbb58d27dd91989f1
I20260807 08:51:59.284832 6353 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e64815bd02294e9c9dc880fc2fb5fbdf: loading cluster ID for follower catalog manager: success
I20260807 08:51:59.285583 6353 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e64815bd02294e9c9dc880fc2fb5fbdf: acquiring CA information for follower catalog manager: success
I20260807 08:51:59.285836 6353 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e64815bd02294e9c9dc880fc2fb5fbdf: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 08:51:59.336819 6460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:59.334630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39068 (local address 127.4.159.189:40563)
0807 08:51:59.334739 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:59.334741 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:59.334777 (+ 36us) server_negotiation.cc:407] Connection header received
0807 08:51:59.334848 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:59.334849 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:59.334898 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:59.334968 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:51:59.335080 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:59.335550 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:59.336239 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:59.336417 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:59.336456 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:59.336561 (+ 105us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:51:59.336673 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:51:59.336727 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:59.336740 (+ 13us) server_negotiation.cc:299] Negotiation successful
0807 08:51:59.336756 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86,"thread_start_us":61,"threads_started":1}
I20260807 08:51:59.340806 6466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:59.338516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45504 (local address 127.4.159.188:44353)
0807 08:51:59.338623 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:59.338625 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:59.338634 (+ 9us) server_negotiation.cc:407] Connection header received
0807 08:51:59.338711 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:59.338712 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:59.338753 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:59.338814 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:51:59.338903 (+ 89us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:59.339418 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:59.340115 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:59.340312 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:59.340364 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:59.340507 (+ 143us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:51:59.340599 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:51:59.340657 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:59.340707 (+ 50us) server_negotiation.cc:299] Negotiation successful
0807 08:51:59.340726 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84,"thread_start_us":48,"threads_started":1}
I20260807 08:51:59.344587 6466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:59.342309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45510 (local address 127.4.159.188:44353)
0807 08:51:59.342362 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:59.342364 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:59.342469 (+ 105us) server_negotiation.cc:407] Connection header received
0807 08:51:59.342545 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:59.342546 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:59.342555 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:59.342595 (+ 40us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:51:59.342687 (+ 92us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:59.343170 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:59.343965 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:59.344156 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:59.344209 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:59.344370 (+ 161us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:51:59.344400 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:51:59.344450 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:59.344497 (+ 47us) server_negotiation.cc:299] Negotiation successful
0807 08:51:59.344516 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 08:51:59.345156 6473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:59.342208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44242 (local address 127.4.159.190:36953)
0807 08:51:59.342307 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:59.342309 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:59.342464 (+ 155us) server_negotiation.cc:407] Connection header received
0807 08:51:59.342529 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:59.342530 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:59.342579 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:59.342668 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:51:59.342805 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:59.343326 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:59.344296 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:59.344546 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:59.344605 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:59.344837 (+ 232us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:51:59.344931 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:51:59.344961 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:59.345050 (+ 89us) server_negotiation.cc:299] Negotiation successful
0807 08:51:59.345074 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76,"thread_start_us":56,"threads_started":1}
I20260807 08:51:59.345944 6460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:51:59.342182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39076 (local address 127.4.159.189:40563)
0807 08:51:59.342217 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0807 08:51:59.342219 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:51:59.342704 (+ 485us) server_negotiation.cc:407] Connection header received
0807 08:51:59.342729 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:51:59.342730 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:51:59.342738 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:51:59.342789 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:51:59.343538 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:59.344255 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:59.345288 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:51:59.345535 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:51:59.345606 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:51:59.345756 (+ 150us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:51:59.345784 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:51:59.345826 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0807 08:51:59.345859 (+ 33us) server_negotiation.cc:299] Negotiation successful
0807 08:51:59.345879 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 08:51:59.346926 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 5946
I20260807 08:51:59.357801 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 6058
I20260807 08:51:59.365298 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 6164
2026-08-07T08:51:59Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (2847 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786092453486433-4734-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:51:59 dist-test-slave-6zbq krb5kdc[6480](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:51:59 dist-test-slave-6zbq krb5kdc[6480](info): set up 2 sockets
Aug 07 08:51:59 dist-test-slave-6zbq krb5kdc[6480](info): commencing operation
krb5kdc: starting...
W20260807 08:52:00.492576 4734 mini_kdc.cc:121] Time spent starting KDC: real 1.107s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786092453486433-4734-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786092453486433-4734-0/krb5kdc/test-user.keytab.
Aug 07 08:52:00 dist-test-slave-6zbq krb5kdc[6480](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092720, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:52:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:52:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.159.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.159.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.159.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab.
Entry for principal kudu/127.4.159.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab.
I20260807 08:52:00.608443 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786092453486433-4734-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786092453486433-4734-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786092453486433-4734-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.159.190:37535
--webserver_interface=127.4.159.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab
--principal=kudu/127.4.159.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:38161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.159.190:37535,127.4.159.189:37101,127.4.159.188:34189
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:52:00.688900 6499 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 08:52:00.689090 6499 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:52:00.689106 6499 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:52:00.689124 6499 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:52:00.690171 6499 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:52:00.690202 6499 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:52:00.690214 6499 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:52:00.690225 6499 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:52:00.690239 6499 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:52:00.691710 6499 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:38161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786092453486433-4734-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786092453486433-4734-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.159.190:37535,127.4.159.189:37101,127.4.159.188:34189
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.190.keytab
--principal=kudu/127.4.159.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.159.190:37535
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.159.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6499
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786092453486433-4734-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:52:00.691896 6499 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:52:00.692059 6499 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:52:00.696668 6505 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:52:00.696691 6504 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:52:00.696686 6511 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:52:00.697006 6499 server_base.cc:1034] running on GCE node
I20260807 08:52:00.697252 6499 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:52:00.697513 6499 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:52:00.698649 6499 hybrid_clock.cc:648] HybridClock initialized: now 1786092720698634 us; error 29 us; skew 500 ppm
Aug 07 08:52:00 dist-test-slave-6zbq krb5kdc[6480](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092720, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:52:00.700914 6499 init.cc:377] Logged in from keytab as kudu/127.4.159.190@KRBTEST.COM (short username kudu)
I20260807 08:52:00.701309 6499 webserver.cc:460] Webserver started at http://127.4.159.190:35539/ using document root <none> and password file <none>
I20260807 08:52:00.701488 6499 fs_manager.cc:359] Metadata directory not provided
I20260807 08:52:00.701525 6499 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:52:00.701613 6499 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:52:00.702600 6499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786092453486433-4734-0/minicluster-data/master-0/data/instance:
uuid: "45d8d6472eab48e5a7b79e970e5e1580"
format_stamp: "Formatted at 2026-08-07 08:52:00 on dist-test-slave-6zbq"
server_key: "d41f29b0b87869f181ffb6adb4413a91"
server_key_iv: "6ac002754b1c5f1ea0b44231fc782323"
server_key_version: "encryptionkey@0"
I20260807 08:52:00.702880 6499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786092453486433-4734-0/minicluster-data/master-0/wal/instance:
uuid: "45d8d6472eab48e5a7b79e970e5e1580"
format_stamp: "Formatted at 2026-08-07 08:52:00 on dist-test-slave-6zbq"
server_key: "d41f29b0b87869f181ffb6adb4413a91"
server_key_iv: "6ac002754b1c5f1ea0b44231fc782323"
server_key_version: "encryptionkey@0"
I20260807 08:52:00.704053 6499 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:52:00.704663 6520 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:52:00.704795 6499 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 08:52:00.704870 6499 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786092453486433-4734-0/minicluster-data/master-0/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786092453486433-4734-0/minicluster-data/master-0/wal
uuid: "45d8d6472eab48e5a7b79e970e5e1580"
format_stamp: "Formatted at 2026-08-07 08:52:00 on dist-test-slave-6zbq"
server_key: "d41f29b0b87869f181ffb6adb4413a91"
server_key_iv: "6ac002754b1c5f1ea0b44231fc782323"
server_key_version: "encryptionkey@0"
I20260807 08:52:00.704934 6499 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786092453486433-4734-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786092453486433-4734-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786092453486433-4734-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:52:00.723404 6499 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:52:00.725291 6499 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:52:00.725409 6499 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:52:00.729099 6499 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.190:37535
I20260807 08:52:00.729130 6600 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.190:37535 every 8 connection(s)
I20260807 08:52:00.729439 6499 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786092453486433-4734-0/minicluster-data/master-0/data/info.pb
I20260807 08:52:00.730329 6601 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:52:00.731761 6601 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.159.190" port: 37535 } has no permanent_uuid. Determining permanent_uuid...
I20260807 08:52:00.733469 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 6499
I20260807 08:52:00.733547 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786092453486433-4734-0/minicluster-data/master-0/wal/instance
I20260807 08:52:00.733716 4734 external_mini_cluster.cc:1428] Setting key fe35039a925243dbabd59c879e6b10bb
Aug 07 08:52:00 dist-test-slave-6zbq krb5kdc[6480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092720, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.190@KRBTEST.COM for kudu/127.4.159.190@KRBTEST.COM
I20260807 08:52:00.740201 6604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:52:00.732384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45360 (local address 127.4.159.190:37535)
0807 08:52:00.732514 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0807 08:52:00.732521 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 08:52:00.732594 (+ 73us) server_negotiation.cc:407] Connection header received
0807 08:52:00.733053 (+ 459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:52:00.733054 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:52:00.733070 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:52:00.733157 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:52:00.733669 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:52:00.734504 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:52:00.735312 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:52:00.735598 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:52:00.735697 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:52:00.737512 (+ 1815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:52:00.737520 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:52:00.737520 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:52:00.737527 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:52:00.738889 (+ 1362us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:52:00.739205 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:52:00.739208 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:52:00.739208 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:52:00.739276 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:52:00.739462 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:52:00.739463 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:52:00.739463 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:52:00.739642 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:52:00.739718 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 08:52:00.739884 (+ 166us) server_negotiation.cc:299] Negotiation successful
0807 08:52:00.739983 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":64,"threads_started":1}
I20260807 08:52:00.740203 6603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:52:00.732337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:37535 (local address 127.0.0.1:45360)
0807 08:52:00.732494 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0807 08:52:00.732531 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 08:52:00.732946 (+ 415us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:52:00.733169 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:52:00.733171 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:52:00.733404 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:52:00.733570 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:52:00.733576 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:52:00.734590 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:52:00.734591 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:52:00.735234 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:52:00.735236 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:52:00.735688 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:52:00.735690 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:52:00.735752 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:52:00.736187 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:52:00.736195 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:52:00.737443 (+ 1248us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:52:00.738957 (+ 1514us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:52:00.738963 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:52:00.738965 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:52:00.739137 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:52:00.739330 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:52:00.739331 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:52:00.739332 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:52:00.739410 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:52:00.739702 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:52:00.739707 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:52:00.739765 (+ 58us) client_negotiation.cc:769] Sending connection context
0807 08:52:00.739844 (+ 79us) client_negotiation.cc:240] Negotiation successful
0807 08:52:00.740055 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"mutex_wait_us":84,"thread_start_us":64,"threads_started":1}
I20260807 08:52:00.740818 6601 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.159.189" port: 37101 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.4.159.189@KRBTEST.COM; defaulting to no policy
I20260807 08:52:00.741115 6603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:52:00.741001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.189:37101 (local address 127.0.0.1:36972)
0807 08:52:00.741038 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0807 08:52:00.741090 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.189:37101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260807 08:52:00.741245 6521 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.159.189:37101: connect: Connection refused (error 111)
I20260807 08:52:00.741600 6603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:52:00.741516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.189:37101 (local address 127.0.0.1:36980)
0807 08:52:00.741555 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0807 08:52:00.741578 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.189:37101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260807 08:52:00.741690 6601 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.159.189:37101: Network error: Client connection negotiation failed: client connection to 127.4.159.189:37101: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.159.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.159.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.189.keytab.
Entry for principal kudu/127.4.159.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.189.keytab.
I20260807 08:52:00.754874 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786092453486433-4734-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786092453486433-4734-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786092453486433-4734-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786092453486433-4734-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.159.189:37101
--webserver_interface=127.4.159.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.189.keytab
--principal=kudu/127.4.159.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:38161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.159.190:37535,127.4.159.189:37101,127.4.159.188:34189
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 08:52:00.790874 6601 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.159.189" port: 37101 } attempt: 1
I20260807 08:52:00.791263 6603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:52:00.791158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.189:37101 (local address 127.0.0.1:36984)
0807 08:52:00.791208 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0807 08:52:00.791236 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.189:37101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260807 08:52:00.791550 6603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:52:00.791468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.189:37101 (local address 127.0.0.1:36992)
0807 08:52:00.791508 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0807 08:52:00.791531 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.189:37101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260807 08:52:00.791637 6601 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.159.189:37101: Network error: Client connection negotiation failed: client connection to 127.4.159.189:37101: connect: Connection refused (error 111)
W20260807 08:52:00.832650 6611 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 08:52:00.832849 6611 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:52:00.832875 6611 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:52:00.832901 6611 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:52:00.833993 6611 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:52:00.834033 6611 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:52:00.834051 6611 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:52:00.834066 6611 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:52:00.834084 6611 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:52:00.835553 6611 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:38161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786092453486433-4734-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786092453486433-4734-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.4.159.190:37535,127.4.159.189:37101,127.4.159.188:34189
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.189.keytab
--principal=kudu/127.4.159.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.159.189:37101
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786092453486433-4734-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.4.159.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6611
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786092453486433-4734-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:52:00.835799 6611 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:52:00.835994 6611 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:52:00.840304 6619 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:52:00.840405 6623 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:52:00.840408 6617 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:52:00.840515 6611 server_base.cc:1034] running on GCE node
I20260807 08:52:00.840799 6611 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:52:00.841037 6611 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:52:00.842216 6611 hybrid_clock.cc:648] HybridClock initialized: now 1786092720842203 us; error 26 us; skew 500 ppm
Aug 07 08:52:00 dist-test-slave-6zbq krb5kdc[6480](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092720, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:52:00.844512 6611 init.cc:377] Logged in from keytab as kudu/127.4.159.189@KRBTEST.COM (short username kudu)
I20260807 08:52:00.844879 6611 webserver.cc:460] Webserver started at http://127.4.159.189:37425/ using document root <none> and password file <none>
I20260807 08:52:00.845041 6611 fs_manager.cc:359] Metadata directory not provided
I20260807 08:52:00.845078 6611 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:52:00.845165 6611 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:52:00.846071 6611 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786092453486433-4734-0/minicluster-data/master-1/data/instance:
uuid: "ae79d9ae8bc246a68cc6feed44f46275"
format_stamp: "Formatted at 2026-08-07 08:52:00 on dist-test-slave-6zbq"
server_key: "95077fd794b54743005ea3683678225c"
server_key_iv: "1b5cef810461dc62c423b071c45cf44e"
server_key_version: "encryptionkey@0"
I20260807 08:52:00.846395 6611 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786092453486433-4734-0/minicluster-data/master-1/wal/instance:
uuid: "ae79d9ae8bc246a68cc6feed44f46275"
format_stamp: "Formatted at 2026-08-07 08:52:00 on dist-test-slave-6zbq"
server_key: "95077fd794b54743005ea3683678225c"
server_key_iv: "1b5cef810461dc62c423b071c45cf44e"
server_key_version: "encryptionkey@0"
I20260807 08:52:00.847532 6611 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 08:52:00.848136 6634 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:52:00.848291 6611 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 08:52:00.848372 6611 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786092453486433-4734-0/minicluster-data/master-1/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786092453486433-4734-0/minicluster-data/master-1/wal
uuid: "ae79d9ae8bc246a68cc6feed44f46275"
format_stamp: "Formatted at 2026-08-07 08:52:00 on dist-test-slave-6zbq"
server_key: "95077fd794b54743005ea3683678225c"
server_key_iv: "1b5cef810461dc62c423b071c45cf44e"
server_key_version: "encryptionkey@0"
I20260807 08:52:00.848433 6611 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786092453486433-4734-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786092453486433-4734-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786092453486433-4734-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:52:00.859773 6601 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.159.189" port: 37101 } attempt: 2
I20260807 08:52:00.860121 6603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:52:00.859994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.189:37101 (local address 127.0.0.1:36996)
0807 08:52:00.860047 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0807 08:52:00.860085 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.189:37101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260807 08:52:00.860422 6603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:52:00.860345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.189:37101 (local address 127.0.0.1:37004)
0807 08:52:00.860382 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0807 08:52:00.860405 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.189:37101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260807 08:52:00.860504 6601 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.159.189:37101: Network error: Client connection negotiation failed: client connection to 127.4.159.189:37101: connect: Connection refused (error 111)
I20260807 08:52:00.866333 6611 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:52:00.867961 6611 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:52:00.868079 6611 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:52:00.871712 6611 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.189:37101
I20260807 08:52:00.871755 6715 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.189:37101 every 8 connection(s)
I20260807 08:52:00.872030 6611 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786092453486433-4734-0/minicluster-data/master-1/data/info.pb
I20260807 08:52:00.872897 6716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:52:00.874262 6716 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.159.190" port: 37535 } has no permanent_uuid. Determining permanent_uuid...
I20260807 08:52:00.878902 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 6611
I20260807 08:52:00.878981 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786092453486433-4734-0/minicluster-data/master-1/wal/instance
I20260807 08:52:00.879153 4734 external_mini_cluster.cc:1428] Setting key bf2d55fdbe9f6d692a7489421c520876
Aug 07 08:52:00 dist-test-slave-6zbq krb5kdc[6480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092720, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.189@KRBTEST.COM for kudu/127.4.159.190@KRBTEST.COM
I20260807 08:52:00.882020 6604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:52:00.874880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45374 (local address 127.4.159.190:37535)
0807 08:52:00.874949 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0807 08:52:00.874951 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:52:00.875103 (+ 152us) server_negotiation.cc:407] Connection header received
0807 08:52:00.875530 (+ 427us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:52:00.875531 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:52:00.875540 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:52:00.875587 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:52:00.876096 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:52:00.876549 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:52:00.877429 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:52:00.877606 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:52:00.877649 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:52:00.879683 (+ 2034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:52:00.879685 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:52:00.879685 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:52:00.879692 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:52:00.880820 (+ 1128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:52:00.881189 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:52:00.881191 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:52:00.881191 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:52:00.881251 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:52:00.881460 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:52:00.881461 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:52:00.881461 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:52:00.881572 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:52:00.881627 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 08:52:00.881889 (+ 262us) server_negotiation.cc:299] Negotiation successful
0807 08:52:00.881960 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 08:52:00.882208 6719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:52:00.874854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:37535 (local address 127.0.0.1:45374)
0807 08:52:00.874997 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0807 08:52:00.875030 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0807 08:52:00.875459 (+ 429us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:52:00.875636 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:52:00.875639 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:52:00.875854 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:52:00.876029 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:52:00.876033 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:52:00.876603 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:52:00.876603 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:52:00.877373 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:52:00.877375 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:52:00.877669 (+ 294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:52:00.877670 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:52:00.877775 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:52:00.878196 (+ 421us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:52:00.878213 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:52:00.879598 (+ 1385us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:52:00.880896 (+ 1298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:52:00.880903 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:52:00.880905 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:52:00.881122 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:52:00.881306 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:52:00.881307 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:52:00.881307 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:52:00.881404 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:52:00.881624 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:52:00.881630 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:52:00.881757 (+ 127us) client_negotiation.cc:769] Sending connection context
0807 08:52:00.881864 (+ 107us) client_negotiation.cc:240] Negotiation successful
0807 08:52:00.882010 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":72,"threads_started":1}
I20260807 08:52:00.882751 6716 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.159.189" port: 37101 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 08:52:00 dist-test-slave-6zbq krb5kdc[6480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092720, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.189@KRBTEST.COM for kudu/127.4.159.189@KRBTEST.COM
WARNING: no policy specified for kudu/127.4.159.188@KRBTEST.COM; defaulting to no policy
I20260807 08:52:00.889464 6719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:52:00.882971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.189:37101 (local address 127.0.0.1:37008)
0807 08:52:00.883014 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0807 08:52:00.883025 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 08:52:00.883066 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:52:00.883289 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:52:00.883290 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:52:00.883437 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:52:00.883493 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:52:00.883495 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:52:00.884320 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:52:00.884321 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:52:00.885066 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:52:00.885069 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:52:00.885517 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:52:00.885518 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:52:00.885591 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:52:00.885893 (+ 302us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:52:00.885898 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:52:00.887116 (+ 1218us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:52:00.888677 (+ 1561us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:52:00.888678 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:52:00.888678 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:52:00.888833 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:52:00.888983 (+ 150us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:52:00.888983 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:52:00.888983 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:52:00.889026 (+ 43us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:52:00.889212 (+ 186us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:52:00.889212 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:52:00.889243 (+ 31us) client_negotiation.cc:769] Sending connection context
0807 08:52:00.889311 (+ 68us) client_negotiation.cc:240] Negotiation successful
0807 08:52:00.889403 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28,"mutex_wait_us":11}
I20260807 08:52:00.889556 6721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:52:00.883033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37008 (local address 127.4.159.189:37101)
0807 08:52:00.883154 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0807 08:52:00.883158 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:52:00.883173 (+ 15us) server_negotiation.cc:407] Connection header received
0807 08:52:00.883190 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:52:00.883192 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:52:00.883201 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:52:00.883263 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:52:00.883560 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:52:00.884225 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:52:00.885140 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:52:00.885450 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:52:00.885498 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:52:00.887197 (+ 1699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:52:00.887204 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:52:00.887204 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:52:00.887210 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:52:00.888609 (+ 1399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:52:00.888892 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:52:00.888894 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:52:00.888894 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:52:00.888940 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:52:00.889063 (+ 123us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:52:00.889064 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:52:00.889064 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:52:00.889171 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:52:00.889222 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0807 08:52:00.889333 (+ 111us) server_negotiation.cc:299] Negotiation successful
0807 08:52:00.889450 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":32,"server-negotiator.queue_time_us":99,"thread_start_us":76,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.159.188@KRBTEST.COM" created.
I20260807 08:52:00.890070 6716 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.159.188" port: 34189 } has no permanent_uuid. Determining permanent_uuid...
I20260807 08:52:00.891438 6719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:52:00.891326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.188:34189 (local address 127.0.0.1:41822)
0807 08:52:00.891375 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0807 08:52:00.891418 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.188:34189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260807 08:52:00.891569 6638 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.159.188:34189: connect: Connection refused (error 111)
I20260807 08:52:00.891953 6719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:52:00.891876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.188:34189 (local address 127.0.0.1:41836)
0807 08:52:00.891915 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0807 08:52:00.891938 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.188:34189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260807 08:52:00.892050 6716 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.159.188:34189: Network error: Client connection negotiation failed: client connection to 127.4.159.188:34189: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.159.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.188.keytab.
Entry for principal kudu/127.4.159.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.188.keytab.
I20260807 08:52:00.902523 4734 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQz81E_/build/release/bin/kudu
/tmp/dist-test-taskQz81E_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786092453486433-4734-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786092453486433-4734-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786092453486433-4734-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786092453486433-4734-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.159.188:34189
--webserver_interface=127.4.159.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.188.keytab
--principal=kudu/127.4.159.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.159.148:38161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.159.190:37535,127.4.159.189:37101,127.4.159.188:34189
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786092453486433-4734-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786092453486433-4734-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786092453486433-4734-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 08:52:00.941295 6716 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.159.188" port: 34189 } attempt: 1
I20260807 08:52:00.941704 6719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:52:00.941600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.188:34189 (local address 127.0.0.1:41844)
0807 08:52:00.941649 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0807 08:52:00.941676 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.188:34189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260807 08:52:00.942092 6719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:52:00.941965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.188:34189 (local address 127.0.0.1:41852)
0807 08:52:00.942027 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0807 08:52:00.942058 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.188:34189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260807 08:52:00.942188 6716 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.159.188:34189: Network error: Client connection negotiation failed: client connection to 127.4.159.188:34189: connect: Connection refused (error 111)
I20260807 08:52:00.951633 6601 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.159.189" port: 37101 } attempt: 3
Aug 07 08:52:00 dist-test-slave-6zbq krb5kdc[6480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092720, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.190@KRBTEST.COM for kudu/127.4.159.189@KRBTEST.COM
I20260807 08:52:00.958464 6603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:52:00.951899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.189:37101 (local address 127.0.0.1:37012)
0807 08:52:00.951943 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0807 08:52:00.951953 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 08:52:00.952003 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:52:00.952128 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:52:00.952129 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:52:00.952264 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:52:00.952327 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:52:00.952329 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:52:00.953210 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:52:00.953211 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:52:00.953733 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:52:00.953734 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:52:00.954210 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:52:00.954211 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:52:00.954267 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:52:00.954685 (+ 418us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:52:00.954689 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:52:00.955997 (+ 1308us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:52:00.957578 (+ 1581us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:52:00.957579 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:52:00.957580 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:52:00.957738 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:52:00.957934 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:52:00.957934 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:52:00.957935 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:52:00.957978 (+ 43us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:52:00.958204 (+ 226us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:52:00.958204 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:52:00.958240 (+ 36us) client_negotiation.cc:769] Sending connection context
0807 08:52:00.958294 (+ 54us) client_negotiation.cc:240] Negotiation successful
0807 08:52:00.958398 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23}
I20260807 08:52:00.958595 6721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:52:00.951928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37012 (local address 127.4.159.189:37101)
0807 08:52:00.951989 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0807 08:52:00.951992 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:52:00.952003 (+ 11us) server_negotiation.cc:407] Connection header received
0807 08:52:00.952050 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:52:00.952051 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:52:00.952061 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:52:00.952118 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:52:00.952388 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:52:00.953141 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:52:00.953803 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:52:00.954145 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:52:00.954215 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:52:00.956079 (+ 1864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:52:00.956081 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:52:00.956081 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:52:00.956088 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:52:00.957484 (+ 1396us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:52:00.957806 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:52:00.957807 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:52:00.957808 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:52:00.957876 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:52:00.958025 (+ 149us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:52:00.958025 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:52:00.958026 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:52:00.958150 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:52:00.958214 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 08:52:00.958329 (+ 115us) server_negotiation.cc:299] Negotiation successful
0807 08:52:00.958423 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 08:52:00.959003 6601 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.159.188" port: 34189 } has no permanent_uuid. Determining permanent_uuid...
I20260807 08:52:00.959287 6603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:52:00.959192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.188:34189 (local address 127.0.0.1:41866)
0807 08:52:00.959231 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0807 08:52:00.959263 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.188:34189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260807 08:52:00.959571 6603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:52:00.959509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.188:34189 (local address 127.0.0.1:41880)
0807 08:52:00.959541 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0807 08:52:00.959560 (+ 19us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.188:34189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260807 08:52:00.959648 6601 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.159.188:34189: Network error: Client connection negotiation failed: client connection to 127.4.159.188:34189: connect: Connection refused (error 111)
W20260807 08:52:00.982626 6724 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 08:52:00.982816 6724 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:52:00.982832 6724 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:52:00.982851 6724 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:52:00.983917 6724 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:52:00.983948 6724 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:52:00.983960 6724 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:52:00.983971 6724 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:52:00.983982 6724 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:52:00.985384 6724 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.159.148:38161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786092453486433-4734-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786092453486433-4734-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.4.159.190:37535,127.4.159.189:37101,127.4.159.188:34189
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786092453486433-4734-0/krb5kdc/kudu_127.4.159.188.keytab
--principal=kudu/127.4.159.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.159.188:34189
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786092453486433-4734-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.4.159.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6724
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786092453486433-4734-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 07:54:20 UTC on ccycloud.quasar-zmatsq.root.comops.site
build id 4572006
I20260807 08:52:00.985594 6724 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:52:00.985776 6724 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:52:00.990070 6731 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:52:00.990087 6735 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:52:00.990061 6732 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:52:00.990196 6724 server_base.cc:1034] running on GCE node
I20260807 08:52:00.990448 6724 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:52:00.990717 6724 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:52:00.990789 6601 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.159.188" port: 34189 } attempt: 1
I20260807 08:52:00.991119 6603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:52:00.991007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.188:34189 (local address 127.0.0.1:41892)
0807 08:52:00.991054 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0807 08:52:00.991081 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.188:34189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260807 08:52:00.991382 6603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:52:00.991333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.188:34189 (local address 127.0.0.1:41908)
0807 08:52:00.991352 (+ 19us) negotiation.cc:107] Waiting for socket to connect
0807 08:52:00.991370 (+ 18us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.188:34189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":11}
W20260807 08:52:00.991449 6601 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.159.188:34189: Network error: Client connection negotiation failed: client connection to 127.4.159.188:34189: connect: Connection refused (error 111)
I20260807 08:52:00.991854 6724 hybrid_clock.cc:648] HybridClock initialized: now 1786092720991843 us; error 29 us; skew 500 ppm
Aug 07 08:52:00 dist-test-slave-6zbq krb5kdc[6480](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092720, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:52:00.994001 6724 init.cc:377] Logged in from keytab as kudu/127.4.159.188@KRBTEST.COM (short username kudu)
I20260807 08:52:00.994421 6724 webserver.cc:460] Webserver started at http://127.4.159.188:39819/ using document root <none> and password file <none>
I20260807 08:52:00.994617 6724 fs_manager.cc:359] Metadata directory not provided
I20260807 08:52:00.994665 6724 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:52:00.994781 6724 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:52:00.995723 6724 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786092453486433-4734-0/minicluster-data/master-2/data/instance:
uuid: "e4dfecedacc248499206c038bcd1eeed"
format_stamp: "Formatted at 2026-08-07 08:52:00 on dist-test-slave-6zbq"
server_key: "1bd895f4336b4bff829b779fe6e70b7c"
server_key_iv: "ae597c4cf15e9dfc70f2321f1669938b"
server_key_version: "encryptionkey@0"
I20260807 08:52:00.996024 6724 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786092453486433-4734-0/minicluster-data/master-2/wal/instance:
uuid: "e4dfecedacc248499206c038bcd1eeed"
format_stamp: "Formatted at 2026-08-07 08:52:00 on dist-test-slave-6zbq"
server_key: "1bd895f4336b4bff829b779fe6e70b7c"
server_key_iv: "ae597c4cf15e9dfc70f2321f1669938b"
server_key_version: "encryptionkey@0"
I20260807 08:52:00.997180 6724 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 08:52:00.997782 6744 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:52:00.997921 6724 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 08:52:00.997997 6724 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786092453486433-4734-0/minicluster-data/master-2/data,/tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786092453486433-4734-0/minicluster-data/master-2/wal
uuid: "e4dfecedacc248499206c038bcd1eeed"
format_stamp: "Formatted at 2026-08-07 08:52:00 on dist-test-slave-6zbq"
server_key: "1bd895f4336b4bff829b779fe6e70b7c"
server_key_iv: "ae597c4cf15e9dfc70f2321f1669938b"
server_key_version: "encryptionkey@0"
I20260807 08:52:00.998055 6724 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786092453486433-4734-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786092453486433-4734-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786092453486433-4734-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:52:01.010313 6716 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.159.188" port: 34189 } attempt: 2
I20260807 08:52:01.010689 6719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:52:01.010563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.188:34189 (local address 127.0.0.1:41920)
0807 08:52:01.010620 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0807 08:52:01.010657 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.188:34189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260807 08:52:01.010988 6719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:52:01.010914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.188:34189 (local address 127.0.0.1:41934)
0807 08:52:01.010950 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0807 08:52:01.010972 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.159.188:34189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260807 08:52:01.011070 6716 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.159.188:34189: Network error: Client connection negotiation failed: client connection to 127.4.159.188:34189: connect: Connection refused (error 111)
I20260807 08:52:01.027789 6724 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:52:01.029803 6724 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:52:01.029927 6724 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:52:01.033519 6724 rpc_server.cc:307] RPC server started. Bound to: 127.4.159.188:34189
I20260807 08:52:01.033557 6817 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.159.188:34189 every 8 connection(s)
I20260807 08:52:01.033852 6724 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786092453486433-4734-0/minicluster-data/master-2/data/info.pb
I20260807 08:52:01.034794 6818 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:52:01.036206 6818 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.159.190" port: 37535 } has no permanent_uuid. Determining permanent_uuid...
I20260807 08:52:01.036711 4734 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQz81E_/build/release/bin/kudu as pid 6724
I20260807 08:52:01.036811 4734 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQz81E_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786092453486433-4734-0/minicluster-data/master-2/wal/instance
I20260807 08:52:01.037015 4734 external_mini_cluster.cc:1428] Setting key 31f2bfde194161d5a8b15db5cccd2156
Aug 07 08:52:01 dist-test-slave-6zbq krb5kdc[6480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092720, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.188@KRBTEST.COM for kudu/127.4.159.190@KRBTEST.COM
Aug 07 08:52:01 dist-test-slave-6zbq krb5kdc[6480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092720, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.159.188@KRBTEST.COM
I20260807 08:52:01.045686 6604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:52:01.036815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45382 (local address 127.4.159.190:37535)
0807 08:52:01.036871 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0807 08:52:01.036873 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:52:01.037115 (+ 242us) server_negotiation.cc:407] Connection header received
0807 08:52:01.037701 (+ 586us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:52:01.037703 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:52:01.037713 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:52:01.037770 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:52:01.038445 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:52:01.039181 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:52:01.040263 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:52:01.040455 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:52:01.040519 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:52:01.042676 (+ 2157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:52:01.042678 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:52:01.042679 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:52:01.042686 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:52:01.044556 (+ 1870us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:52:01.044911 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:52:01.044912 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:52:01.044912 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:52:01.044969 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:52:01.045186 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:52:01.045186 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:52:01.045187 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:52:01.045288 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:52:01.045339 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0807 08:52:01.045516 (+ 177us) server_negotiation.cc:299] Negotiation successful
0807 08:52:01.045594 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260807 08:52:01.045868 6826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:52:01.036791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.190:37535 (local address 127.0.0.1:45382)
0807 08:52:01.036959 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0807 08:52:01.037027 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0807 08:52:01.037606 (+ 579us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:52:01.037797 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:52:01.037799 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:52:01.038113 (+ 314us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:52:01.038364 (+ 251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:52:01.038370 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:52:01.039255 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:52:01.039256 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:52:01.040183 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:52:01.040185 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:52:01.040548 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:52:01.040549 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:52:01.040642 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:52:01.041069 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:52:01.041092 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:52:01.042592 (+ 1500us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:52:01.044639 (+ 2047us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:52:01.044646 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:52:01.044647 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:52:01.044837 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:52:01.045025 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:52:01.045026 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:52:01.045026 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:52:01.045127 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:52:01.045344 (+ 217us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:52:01.045349 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:52:01.045405 (+ 56us) client_negotiation.cc:769] Sending connection context
0807 08:52:01.045490 (+ 85us) client_negotiation.cc:240] Negotiation successful
0807 08:52:01.045721 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"mutex_wait_us":102,"thread_start_us":83,"threads_started":1}
I20260807 08:52:01.045868 6830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:52:01.037402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41942 (local address 127.4.159.188:34189)
0807 08:52:01.037520 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0807 08:52:01.037524 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:52:01.037559 (+ 35us) server_negotiation.cc:407] Connection header received
0807 08:52:01.037665 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:52:01.037672 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:52:01.037695 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:52:01.037785 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:52:01.038503 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:52:01.039548 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:52:01.040466 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:52:01.040798 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:52:01.040862 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:52:01.043101 (+ 2239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:52:01.043109 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:52:01.043110 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:52:01.043116 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:52:01.044499 (+ 1383us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:52:01.044825 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:52:01.044828 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:52:01.044829 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:52:01.044918 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:52:01.045111 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:52:01.045112 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:52:01.045112 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:52:01.045335 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:52:01.045465 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0807 08:52:01.045533 (+ 68us) server_negotiation.cc:299] Negotiation successful
0807 08:52:01.045648 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90,"thread_start_us":66,"threads_started":1}
I20260807 08:52:01.046530 6818 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.159.189" port: 37101 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 08:52:01 dist-test-slave-6zbq krb5kdc[6480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092720, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.188@KRBTEST.COM for kudu/127.4.159.189@KRBTEST.COM
Aug 07 08:52:01 dist-test-slave-6zbq krb5kdc[6480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092720, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.159.190@KRBTEST.COM
I20260807 08:52:01.052985 6826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:52:01.046777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.189:37101 (local address 127.0.0.1:37028)
0807 08:52:01.046815 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0807 08:52:01.046823 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0807 08:52:01.046856 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:52:01.046976 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:52:01.046976 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:52:01.047080 (+ 104us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:52:01.047107 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:52:01.047108 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:52:01.047721 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:52:01.047722 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:52:01.048192 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:52:01.048194 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:52:01.048542 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:52:01.048543 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:52:01.048601 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:52:01.049021 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:52:01.049024 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:52:01.050212 (+ 1188us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:52:01.052055 (+ 1843us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:52:01.052056 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:52:01.052056 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:52:01.052256 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:52:01.052429 (+ 173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:52:01.052429 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:52:01.052430 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:52:01.052493 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:52:01.052722 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:52:01.052722 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:52:01.052764 (+ 42us) client_negotiation.cc:769] Sending connection context
0807 08:52:01.052828 (+ 64us) client_negotiation.cc:240] Negotiation successful
0807 08:52:01.052906 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260807 08:52:01.053027 6721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:52:01.046830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37028 (local address 127.4.159.189:37101)
0807 08:52:01.046880 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0807 08:52:01.046882 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:52:01.046889 (+ 7us) server_negotiation.cc:407] Connection header received
0807 08:52:01.046912 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:52:01.046913 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:52:01.046922 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:52:01.046976 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:52:01.047166 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:52:01.047649 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:52:01.048263 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:52:01.048479 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:52:01.048537 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:52:01.050291 (+ 1754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:52:01.050292 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:52:01.050292 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:52:01.050298 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:52:01.051974 (+ 1676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:52:01.052324 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:52:01.052325 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:52:01.052325 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:52:01.052371 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:52:01.052548 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:52:01.052549 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:52:01.052549 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:52:01.052667 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:52:01.052728 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 08:52:01.052858 (+ 130us) server_negotiation.cc:299] Negotiation successful
0807 08:52:01.052933 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 08:52:01.053314 6818 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.159.188" port: 34189 } has no permanent_uuid. Determining permanent_uuid...
I20260807 08:52:01.053668 6604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:52:01.046652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45386 (local address 127.4.159.190:37535)
0807 08:52:01.046691 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0807 08:52:01.046693 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:52:01.046701 (+ 8us) server_negotiation.cc:407] Connection header received
0807 08:52:01.046739 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:52:01.046740 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:52:01.046750 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:52:01.046792 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:52:01.047275 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:52:01.047685 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:52:01.048286 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:52:01.048462 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:52:01.048506 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:52:01.050984 (+ 2478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:52:01.050986 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:52:01.050986 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:52:01.050993 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:52:01.052394 (+ 1401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:52:01.052718 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:52:01.052719 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:52:01.052719 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:52:01.052772 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:52:01.052954 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:52:01.052955 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:52:01.052955 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:52:01.053074 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:52:01.053218 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0807 08:52:01.053392 (+ 174us) server_negotiation.cc:299] Negotiation successful
0807 08:52:01.053461 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
Aug 07 08:52:01 dist-test-slave-6zbq krb5kdc[6480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092720, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.188@KRBTEST.COM for kudu/127.4.159.188@KRBTEST.COM
Aug 07 08:52:01 dist-test-slave-6zbq krb5kdc[6480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092720, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.159.189@KRBTEST.COM
I20260807 08:52:01.059907 6826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:52:01.053523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.188:34189 (local address 127.0.0.1:41946)
0807 08:52:01.053741 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0807 08:52:01.053750 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 08:52:01.053808 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:52:01.053961 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:52:01.053962 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:52:01.054111 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:52:01.054156 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:52:01.054158 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:52:01.054848 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:52:01.054849 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:52:01.055335 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:52:01.055338 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:52:01.055671 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:52:01.055671 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:52:01.055724 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:52:01.056047 (+ 323us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:52:01.056052 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:52:01.057266 (+ 1214us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:52:01.058927 (+ 1661us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:52:01.058928 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:52:01.058929 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:52:01.059155 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:52:01.059330 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:52:01.059331 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:52:01.059331 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:52:01.059393 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:52:01.059584 (+ 191us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:52:01.059585 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:52:01.059628 (+ 43us) client_negotiation.cc:769] Sending connection context
0807 08:52:01.059696 (+ 68us) client_negotiation.cc:240] Negotiation successful
0807 08:52:01.059792 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205}
I20260807 08:52:01.059944 6830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:52:01.053617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41946 (local address 127.4.159.188:34189)
0807 08:52:01.053648 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0807 08:52:01.053650 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:52:01.053788 (+ 138us) server_negotiation.cc:407] Connection header received
0807 08:52:01.053876 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:52:01.053877 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:52:01.053888 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:52:01.053946 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:52:01.054216 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:52:01.054778 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:52:01.055415 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:52:01.055612 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:52:01.055658 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:52:01.057349 (+ 1691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:52:01.057350 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:52:01.057350 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:52:01.057356 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:52:01.058853 (+ 1497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:52:01.059220 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:52:01.059221 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:52:01.059221 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:52:01.059275 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:52:01.059445 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:52:01.059445 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:52:01.059446 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:52:01.059535 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:52:01.059576 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0807 08:52:01.059712 (+ 136us) server_negotiation.cc:299] Negotiation successful
0807 08:52:01.059835 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":64,"server-negotiator.queue_time_us":16}
I20260807 08:52:01.060711 6721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:52:01.054209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37036 (local address 127.4.159.189:37101)
0807 08:52:01.054246 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0807 08:52:01.054247 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 08:52:01.054254 (+ 7us) server_negotiation.cc:407] Connection header received
0807 08:52:01.054431 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:52:01.054432 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:52:01.054441 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:52:01.054494 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:52:01.054896 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:52:01.055290 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:52:01.056021 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:52:01.056221 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:52:01.056274 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:52:01.058333 (+ 2059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:52:01.058334 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:52:01.058335 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:52:01.058363 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:52:01.059668 (+ 1305us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:52:01.060034 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:52:01.060036 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:52:01.060036 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:52:01.060090 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:52:01.060292 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:52:01.060293 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:52:01.060293 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:52:01.060398 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:52:01.060547 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0807 08:52:01.060579 (+ 32us) server_negotiation.cc:299] Negotiation successful
0807 08:52:01.060642 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260807 08:52:01.061408 6818 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4dfecedacc248499206c038bcd1eeed: Bootstrap starting.
I20260807 08:52:01.062170 6818 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e4dfecedacc248499206c038bcd1eeed: Neither blocks nor log segments found. Creating new log.
I20260807 08:52:01.062594 6818 log.cc:824] T 00000000000000000000000000000000 P e4dfecedacc248499206c038bcd1eeed: Log is configured to *not* fsync() on all Append() calls
I20260807 08:52:01.063328 6818 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4dfecedacc248499206c038bcd1eeed: No bootstrap required, opened a new log
I20260807 08:52:01.063706 4734 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 08:52:01.064919 6818 raft_consensus.cc:348] T 00000000000000000000000000000000 P e4dfecedacc248499206c038bcd1eeed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45d8d6472eab48e5a7b79e970e5e1580" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 37535 } } peers { permanent_uuid: "ae79d9ae8bc246a68cc6feed44f46275" member_type: VOTER last_known_addr { host: "127.4.159.189" port: 37101 } } peers { permanent_uuid: "e4dfecedacc248499206c038bcd1eeed" member_type: VOTER last_known_addr { host: "127.4.159.188" port: 34189 } }
I20260807 08:52:01.065050 6818 raft_consensus.cc:729] T 00000000000000000000000000000000 P e4dfecedacc248499206c038bcd1eeed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4dfecedacc248499206c038bcd1eeed, State: Initialized, Role: FOLLOWER
I20260807 08:52:01.065135 6818 consensus_queue.cc:260] T 00000000000000000000000000000000 P e4dfecedacc248499206c038bcd1eeed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45d8d6472eab48e5a7b79e970e5e1580" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 37535 } } peers { permanent_uuid: "ae79d9ae8bc246a68cc6feed44f46275" member_type: VOTER last_known_addr { host: "127.4.159.189" port: 37101 } } peers { permanent_uuid: "e4dfecedacc248499206c038bcd1eeed" member_type: VOTER last_known_addr { host: "127.4.159.188" port: 34189 } }
I20260807 08:52:01.064503 6559 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "45d8d6472eab48e5a7b79e970e5e1580"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45386
I20260807 08:52:01.065361 6832 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4dfecedacc248499206c038bcd1eeed [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45d8d6472eab48e5a7b79e970e5e1580" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 37535 } } peers { permanent_uuid: "ae79d9ae8bc246a68cc6feed44f46275" member_type: VOTER last_known_addr { host: "127.4.159.189" port: 37101 } } peers { permanent_uuid: "e4dfecedacc248499206c038bcd1eeed" member_type: VOTER last_known_addr { host: "127.4.159.188" port: 34189 } } }
I20260807 08:52:01.065419 6818 sys_catalog.cc:564] T 00000000000000000000000000000000 P e4dfecedacc248499206c038bcd1eeed [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:52:01.065466 6832 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4dfecedacc248499206c038bcd1eeed [sys.catalog]: This master's current role is: FOLLOWER
I20260807 08:52:01.066569 6601 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.159.188" port: 34189 } attempt: 2
W20260807 08:52:01.067404 6848 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e4dfecedacc248499206c038bcd1eeed: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 08:52:01.067464 6848 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 08:52:01.067263 6724 master_runner.cc:186] Error getting master registration for 127.4.159.190:37535: OK, instance_id { permanent_uuid: "45d8d6472eab48e5a7b79e970e5e1580" instance_seqno: 1786092720728117 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 08:52:01.069905 6724 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Aug 07 08:52:01 dist-test-slave-6zbq krb5kdc[6480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092720, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.190@KRBTEST.COM for kudu/127.4.159.188@KRBTEST.COM
I20260807 08:52:01.073534 6603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:52:01.066799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.188:34189 (local address 127.0.0.1:41962)
0807 08:52:01.066846 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0807 08:52:01.066855 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 08:52:01.066889 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:52:01.067052 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:52:01.067053 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:52:01.067209 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:52:01.067249 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:52:01.067251 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:52:01.068051 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:52:01.068052 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:52:01.068819 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:52:01.068822 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:52:01.069302 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:52:01.069303 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:52:01.069358 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:52:01.069750 (+ 392us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:52:01.069754 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:52:01.070985 (+ 1231us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:52:01.072551 (+ 1566us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:52:01.072552 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:52:01.072553 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:52:01.072770 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:52:01.072942 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:52:01.072942 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:52:01.072943 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:52:01.073011 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:52:01.073249 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:52:01.073250 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:52:01.073291 (+ 41us) client_negotiation.cc:769] Sending connection context
0807 08:52:01.073353 (+ 62us) client_negotiation.cc:240] Negotiation successful
0807 08:52:01.073443 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260807 08:52:01.073534 6830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:52:01.066816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41962 (local address 127.4.159.188:34189)
0807 08:52:01.066949 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0807 08:52:01.066951 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:52:01.066958 (+ 7us) server_negotiation.cc:407] Connection header received
0807 08:52:01.066975 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:52:01.066976 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:52:01.066984 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:52:01.067030 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:52:01.067329 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:52:01.067960 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:52:01.068907 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:52:01.069196 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:52:01.069254 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:52:01.071073 (+ 1819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:52:01.071074 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:52:01.071074 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:52:01.071080 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:52:01.072491 (+ 1411us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:52:01.072822 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:52:01.072823 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:52:01.072823 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:52:01.072889 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:52:01.073063 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:52:01.073064 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:52:01.073064 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:52:01.073185 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:52:01.073241 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 08:52:01.073388 (+ 147us) server_negotiation.cc:299] Negotiation successful
0807 08:52:01.073451 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115}
I20260807 08:52:01.074934 6601 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 45d8d6472eab48e5a7b79e970e5e1580: Bootstrap starting.
I20260807 08:52:01.075515 6601 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 45d8d6472eab48e5a7b79e970e5e1580: Neither blocks nor log segments found. Creating new log.
I20260807 08:52:01.075752 6601 log.cc:824] T 00000000000000000000000000000000 P 45d8d6472eab48e5a7b79e970e5e1580: Log is configured to *not* fsync() on all Append() calls
I20260807 08:52:01.076443 6601 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 45d8d6472eab48e5a7b79e970e5e1580: No bootstrap required, opened a new log
I20260807 08:52:01.077380 6601 raft_consensus.cc:348] T 00000000000000000000000000000000 P 45d8d6472eab48e5a7b79e970e5e1580 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45d8d6472eab48e5a7b79e970e5e1580" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 37535 } } peers { permanent_uuid: "ae79d9ae8bc246a68cc6feed44f46275" member_type: VOTER last_known_addr { host: "127.4.159.189" port: 37101 } } peers { permanent_uuid: "e4dfecedacc248499206c038bcd1eeed" member_type: VOTER last_known_addr { host: "127.4.159.188" port: 34189 } }
I20260807 08:52:01.077490 6601 raft_consensus.cc:729] T 00000000000000000000000000000000 P 45d8d6472eab48e5a7b79e970e5e1580 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45d8d6472eab48e5a7b79e970e5e1580, State: Initialized, Role: FOLLOWER
I20260807 08:52:01.077565 6601 consensus_queue.cc:260] T 00000000000000000000000000000000 P 45d8d6472eab48e5a7b79e970e5e1580 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45d8d6472eab48e5a7b79e970e5e1580" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 37535 } } peers { permanent_uuid: "ae79d9ae8bc246a68cc6feed44f46275" member_type: VOTER last_known_addr { host: "127.4.159.189" port: 37101 } } peers { permanent_uuid: "e4dfecedacc248499206c038bcd1eeed" member_type: VOTER last_known_addr { host: "127.4.159.188" port: 34189 } }
I20260807 08:52:01.077798 6851 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45d8d6472eab48e5a7b79e970e5e1580 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45d8d6472eab48e5a7b79e970e5e1580" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 37535 } } peers { permanent_uuid: "ae79d9ae8bc246a68cc6feed44f46275" member_type: VOTER last_known_addr { host: "127.4.159.189" port: 37101 } } peers { permanent_uuid: "e4dfecedacc248499206c038bcd1eeed" member_type: VOTER last_known_addr { host: "127.4.159.188" port: 34189 } } }
I20260807 08:52:01.077854 6601 sys_catalog.cc:564] T 00000000000000000000000000000000 P 45d8d6472eab48e5a7b79e970e5e1580 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:52:01.077877 6851 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45d8d6472eab48e5a7b79e970e5e1580 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 08:52:01.077890 6559 raft_consensus.cc:482] T 00000000000000000000000000000000 P 45d8d6472eab48e5a7b79e970e5e1580 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 08:52:01.077961 6559 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 45d8d6472eab48e5a7b79e970e5e1580 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:52:01.078730 6559 raft_consensus.cc:504] T 00000000000000000000000000000000 P 45d8d6472eab48e5a7b79e970e5e1580 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45d8d6472eab48e5a7b79e970e5e1580" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 37535 } } peers { permanent_uuid: "ae79d9ae8bc246a68cc6feed44f46275" member_type: VOTER last_known_addr { host: "127.4.159.189" port: 37101 } } peers { permanent_uuid: "e4dfecedacc248499206c038bcd1eeed" member_type: VOTER last_known_addr { host: "127.4.159.188" port: 34189 } }
I20260807 08:52:01.078956 6559 leader_election.cc:288] T 00000000000000000000000000000000 P 45d8d6472eab48e5a7b79e970e5e1580 [CANDIDATE]: Term 1 election: Requested vote from peers ae79d9ae8bc246a68cc6feed44f46275 (127.4.159.189:37101), e4dfecedacc248499206c038bcd1eeed (127.4.159.188:34189)
W20260807 08:52:01.079687 6867 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 45d8d6472eab48e5a7b79e970e5e1580: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 08:52:01.079731 6867 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 08:52:01.080490 6784 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "45d8d6472eab48e5a7b79e970e5e1580" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "e4dfecedacc248499206c038bcd1eeed"
I20260807 08:52:01.080605 6784 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e4dfecedacc248499206c038bcd1eeed [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:52:01.081477 6784 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e4dfecedacc248499206c038bcd1eeed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45d8d6472eab48e5a7b79e970e5e1580 in term 1.
I20260807 08:52:01.081702 6522 leader_election.cc:302] T 00000000000000000000000000000000 P 45d8d6472eab48e5a7b79e970e5e1580 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45d8d6472eab48e5a7b79e970e5e1580, e4dfecedacc248499206c038bcd1eeed; no voters:
I20260807 08:52:01.081825 6851 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 45d8d6472eab48e5a7b79e970e5e1580 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:52:01.081972 6851 raft_consensus.cc:686] T 00000000000000000000000000000000 P 45d8d6472eab48e5a7b79e970e5e1580 [term 1 LEADER]: Becoming Leader. State: Replica: 45d8d6472eab48e5a7b79e970e5e1580, State: Running, Role: LEADER
I20260807 08:52:01.082052 6851 consensus_queue.cc:237] T 00000000000000000000000000000000 P 45d8d6472eab48e5a7b79e970e5e1580 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45d8d6472eab48e5a7b79e970e5e1580" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 37535 } } peers { permanent_uuid: "ae79d9ae8bc246a68cc6feed44f46275" member_type: VOTER last_known_addr { host: "127.4.159.189" port: 37101 } } peers { permanent_uuid: "e4dfecedacc248499206c038bcd1eeed" member_type: VOTER last_known_addr { host: "127.4.159.188" port: 34189 } }
I20260807 08:52:01.082500 6851 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45d8d6472eab48e5a7b79e970e5e1580 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 45d8d6472eab48e5a7b79e970e5e1580. Latest consensus state: current_term: 1 leader_uuid: "45d8d6472eab48e5a7b79e970e5e1580" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45d8d6472eab48e5a7b79e970e5e1580" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 37535 } } peers { permanent_uuid: "ae79d9ae8bc246a68cc6feed44f46275" member_type: VOTER last_known_addr { host: "127.4.159.189" port: 37101 } } peers { permanent_uuid: "e4dfecedacc248499206c038bcd1eeed" member_type: VOTER last_known_addr { host: "127.4.159.188" port: 34189 } } }
I20260807 08:52:01.082576 6851 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45d8d6472eab48e5a7b79e970e5e1580 [sys.catalog]: This master's current role is: LEADER
I20260807 08:52:01.083681 6881 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:52:01.083873 6881 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:52:01.082173 6499 master_runner.cc:186] Error getting master registration for 127.4.159.189:37101: OK, instance_id { permanent_uuid: "ae79d9ae8bc246a68cc6feed44f46275" instance_seqno: 1786092720870698 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 08:52:01.084340 6499 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 08:52:01.086535 6784 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e4dfecedacc248499206c038bcd1eeed [term 1 FOLLOWER]: Refusing update from remote peer 45d8d6472eab48e5a7b79e970e5e1580: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:52:01.086826 6851 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 45d8d6472eab48e5a7b79e970e5e1580 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e4dfecedacc248499206c038bcd1eeed" member_type: VOTER last_known_addr { host: "127.4.159.188" port: 34189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:52:01.088011 6832 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4dfecedacc248499206c038bcd1eeed [sys.catalog]: SysCatalogTable state changed. Reason: New leader 45d8d6472eab48e5a7b79e970e5e1580. Latest consensus state: current_term: 1 leader_uuid: "45d8d6472eab48e5a7b79e970e5e1580" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45d8d6472eab48e5a7b79e970e5e1580" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 37535 } } peers { permanent_uuid: "ae79d9ae8bc246a68cc6feed44f46275" member_type: VOTER last_known_addr { host: "127.4.159.189" port: 37101 } } peers { permanent_uuid: "e4dfecedacc248499206c038bcd1eeed" member_type: VOTER last_known_addr { host: "127.4.159.188" port: 34189 } } }
I20260807 08:52:01.088121 6832 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4dfecedacc248499206c038bcd1eeed [sys.catalog]: This master's current role is: FOLLOWER
I20260807 08:52:01.089358 6882 mvcc.cc:205] Tried to move back new op lower bound from 7315835785568075776 to 7315835785552875520. Current Snapshot: MvccSnapshot[applied={T|T < 7315835785568075776}]
I20260807 08:52:01.089653 6851 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45d8d6472eab48e5a7b79e970e5e1580 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "45d8d6472eab48e5a7b79e970e5e1580" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45d8d6472eab48e5a7b79e970e5e1580" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 37535 } } peers { permanent_uuid: "ae79d9ae8bc246a68cc6feed44f46275" member_type: VOTER last_known_addr { host: "127.4.159.189" port: 37101 } } peers { permanent_uuid: "e4dfecedacc248499206c038bcd1eeed" member_type: VOTER last_known_addr { host: "127.4.159.188" port: 34189 } } }
I20260807 08:52:01.089740 6851 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45d8d6472eab48e5a7b79e970e5e1580 [sys.catalog]: This master's current role is: LEADER
I20260807 08:52:01.089843 6604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:52:01.080281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45402 (local address 127.4.159.190:37535)
0807 08:52:01.080411 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0807 08:52:01.080413 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:52:01.080422 (+ 9us) server_negotiation.cc:407] Connection header received
0807 08:52:01.080504 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:52:01.080506 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:52:01.080516 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:52:01.080544 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:52:01.081272 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:52:01.082000 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:52:01.083563 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:52:01.083854 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:52:01.083924 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:52:01.085760 (+ 1836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:52:01.085762 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:52:01.085762 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:52:01.085769 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:52:01.087637 (+ 1868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:52:01.088306 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:52:01.088307 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:52:01.088308 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:52:01.088365 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:52:01.088603 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:52:01.088604 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:52:01.088605 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:52:01.088729 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:52:01.088831 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0807 08:52:01.089545 (+ 714us) server_negotiation.cc:299] Negotiation successful
0807 08:52:01.089619 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111}
I20260807 08:52:01.090799 6832 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4dfecedacc248499206c038bcd1eeed [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "45d8d6472eab48e5a7b79e970e5e1580" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45d8d6472eab48e5a7b79e970e5e1580" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 37535 } } peers { permanent_uuid: "ae79d9ae8bc246a68cc6feed44f46275" member_type: VOTER last_known_addr { host: "127.4.159.189" port: 37101 } } peers { permanent_uuid: "e4dfecedacc248499206c038bcd1eeed" member_type: VOTER last_known_addr { host: "127.4.159.188" port: 34189 } } }
I20260807 08:52:01.090891 6832 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4dfecedacc248499206c038bcd1eeed [sys.catalog]: This master's current role is: FOLLOWER
I20260807 08:52:01.091007 6885 mvcc.cc:205] Tried to move back new op lower bound from 7315835785568075776 to 7315835785552875520. Current Snapshot: MvccSnapshot[applied={T|T < 7315835785568075776}]
I20260807 08:52:01.095364 6881 catalog_manager.cc:1347] Generated new cluster ID: 9688cc2f59b74fdb8eda24c64f97d88f
I20260807 08:52:01.095415 6881 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:52:01.098871 6721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:52:01.080944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37052 (local address 127.4.159.189:37101)
0807 08:52:01.080982 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0807 08:52:01.080984 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:52:01.089334 (+ 8350us) server_negotiation.cc:407] Connection header received
0807 08:52:01.089358 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:52:01.089359 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:52:01.089370 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:52:01.089412 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:52:01.090847 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:52:01.091571 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:52:01.093040 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:52:01.093382 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:52:01.093423 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:52:01.095598 (+ 2175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:52:01.095599 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:52:01.095599 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:52:01.095618 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:52:01.097316 (+ 1698us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:52:01.097872 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:52:01.097873 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:52:01.097874 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:52:01.097940 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:52:01.098191 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:52:01.098192 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:52:01.098192 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:52:01.098322 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:52:01.098425 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0807 08:52:01.098702 (+ 277us) server_negotiation.cc:299] Negotiation successful
0807 08:52:01.098777 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 08:52:01.102177 6716 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.159.188" port: 34189 } attempt: 3
I20260807 08:52:01.112171 6830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:52:01.087564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41964 (local address 127.4.159.188:34189)
0807 08:52:01.087595 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0807 08:52:01.087597 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:52:01.090277 (+ 2680us) server_negotiation.cc:407] Connection header received
0807 08:52:01.090632 (+ 355us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:52:01.090633 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:52:01.090644 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:52:01.090672 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:52:01.094948 (+ 4276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:52:01.095561 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:52:01.103265 (+ 7704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:52:01.103544 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:52:01.103602 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:52:01.105983 (+ 2381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:52:01.105985 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:52:01.105985 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:52:01.105992 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:52:01.107570 (+ 1578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:52:01.109659 (+ 2089us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:52:01.109660 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:52:01.109660 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:52:01.109732 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:52:01.110224 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:52:01.110238 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:52:01.110239 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:52:01.110402 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:52:01.110481 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 08:52:01.112006 (+ 1525us) server_negotiation.cc:299] Negotiation successful
0807 08:52:01.112083 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
Aug 07 08:52:01 dist-test-slave-6zbq krb5kdc[6480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786092720, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.159.189@KRBTEST.COM for kudu/127.4.159.188@KRBTEST.COM
I20260807 08:52:01.116710 6719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:52:01.102586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.159.188:34189 (local address 127.0.0.1:41974)
0807 08:52:01.102629 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0807 08:52:01.102637 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0807 08:52:01.102676 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:52:01.102939 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:52:01.102940 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:52:01.103098 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:52:01.103137 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:52:01.103138 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:52:01.108113 (+ 4975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:52:01.108114 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:52:01.108902 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:52:01.108905 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:52:01.110896 (+ 1991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:52:01.110897 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:52:01.110968 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:52:01.111353 (+ 385us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:52:01.111358 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:52:01.112796 (+ 1438us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:52:01.115426 (+ 2630us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:52:01.115427 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:52:01.115444 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:52:01.115660 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:52:01.115923 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:52:01.115923 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:52:01.115924 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:52:01.116001 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:52:01.116405 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:52:01.116405 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:52:01.116449 (+ 44us) client_negotiation.cc:769] Sending connection context
0807 08:52:01.116512 (+ 63us) client_negotiation.cc:240] Negotiation successful
0807 08:52:01.116612 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260807 08:52:01.117164 6894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:52:01.102464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41974 (local address 127.4.159.188:34189)
0807 08:52:01.102648 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0807 08:52:01.102650 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:52:01.102663 (+ 13us) server_negotiation.cc:407] Connection header received
0807 08:52:01.102801 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:52:01.102802 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:52:01.102812 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:52:01.103210 (+ 398us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:52:01.103274 (+ 64us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:52:01.107938 (+ 4664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:52:01.109079 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:52:01.110710 (+ 1631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:52:01.111869 (+ 1159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:52:01.112907 (+ 1038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:52:01.112909 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:52:01.112909 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:52:01.112937 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:52:01.115329 (+ 2392us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:52:01.115742 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:52:01.115743 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:52:01.115744 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:52:01.115869 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:52:01.116073 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:52:01.116074 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:52:01.116075 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:52:01.116333 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:52:01.116416 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 08:52:01.116573 (+ 157us) server_negotiation.cc:299] Negotiation successful
0807 08:52:01.116736 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":85,"threads_started":1}
I20260807 08:52:01.118907 6716 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae79d9ae8bc246a68cc6feed44f46275: Bootstrap starting.
I20260807 08:52:01.119614 6716 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ae79d9ae8bc246a68cc6feed44f46275: Neither blocks nor log segments found. Creating new log.
I20260807 08:52:01.119894 6716 log.cc:824] T 00000000000000000000000000000000 P ae79d9ae8bc246a68cc6feed44f46275: Log is configured to *not* fsync() on all Append() calls
I20260807 08:52:01.120740 6716 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae79d9ae8bc246a68cc6feed44f46275: No bootstrap required, opened a new log
I20260807 08:52:01.122377 6881 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:52:01.122726 6716 raft_consensus.cc:348] T 00000000000000000000000000000000 P ae79d9ae8bc246a68cc6feed44f46275 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45d8d6472eab48e5a7b79e970e5e1580" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 37535 } } peers { permanent_uuid: "ae79d9ae8bc246a68cc6feed44f46275" member_type: VOTER last_known_addr { host: "127.4.159.189" port: 37101 } } peers { permanent_uuid: "e4dfecedacc248499206c038bcd1eeed" member_type: VOTER last_known_addr { host: "127.4.159.188" port: 34189 } }
I20260807 08:52:01.122862 6716 raft_consensus.cc:729] T 00000000000000000000000000000000 P ae79d9ae8bc246a68cc6feed44f46275 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae79d9ae8bc246a68cc6feed44f46275, State: Initialized, Role: FOLLOWER
I20260807 08:52:01.122943 6716 consensus_queue.cc:260] T 00000000000000000000000000000000 P ae79d9ae8bc246a68cc6feed44f46275 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45d8d6472eab48e5a7b79e970e5e1580" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 37535 } } peers { permanent_uuid: "ae79d9ae8bc246a68cc6feed44f46275" member_type: VOTER last_known_addr { host: "127.4.159.189" port: 37101 } } peers { permanent_uuid: "e4dfecedacc248499206c038bcd1eeed" member_type: VOTER last_known_addr { host: "127.4.159.188" port: 34189 } }
I20260807 08:52:01.123077 6881 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:52:01.123287 6716 sys_catalog.cc:564] T 00000000000000000000000000000000 P ae79d9ae8bc246a68cc6feed44f46275 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:52:01.123349 6672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "45d8d6472eab48e5a7b79e970e5e1580" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "ae79d9ae8bc246a68cc6feed44f46275"
I20260807 08:52:01.123445 6672 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ae79d9ae8bc246a68cc6feed44f46275 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:52:01.124192 6672 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ae79d9ae8bc246a68cc6feed44f46275 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45d8d6472eab48e5a7b79e970e5e1580 in term 1.
I20260807 08:52:01.124366 6671 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ae79d9ae8bc246a68cc6feed44f46275 [term 1 FOLLOWER]: Refusing update from remote peer 45d8d6472eab48e5a7b79e970e5e1580: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:52:01.125159 6890 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 45d8d6472eab48e5a7b79e970e5e1580 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ae79d9ae8bc246a68cc6feed44f46275" member_type: VOTER last_known_addr { host: "127.4.159.189" port: 37101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:52:01.123301 6898 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae79d9ae8bc246a68cc6feed44f46275 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45d8d6472eab48e5a7b79e970e5e1580" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 37535 } } peers { permanent_uuid: "ae79d9ae8bc246a68cc6feed44f46275" member_type: VOTER last_known_addr { host: "127.4.159.189" port: 37101 } } peers { permanent_uuid: "e4dfecedacc248499206c038bcd1eeed" member_type: VOTER last_known_addr { host: "127.4.159.188" port: 34189 } } }
I20260807 08:52:01.125900 6898 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae79d9ae8bc246a68cc6feed44f46275 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 08:52:01.127707 6898 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae79d9ae8bc246a68cc6feed44f46275 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 45d8d6472eab48e5a7b79e970e5e1580. Latest consensus state: current_term: 1 leader_uuid: "45d8d6472eab48e5a7b79e970e5e1580" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45d8d6472eab48e5a7b79e970e5e1580" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 37535 } } peers { permanent_uuid: "ae79d9ae8bc246a68cc6feed44f46275" member_type: VOTER last_known_addr { host: "127.4.159.189" port: 37101 } } peers { permanent_uuid: "e4dfecedacc248499206c038bcd1eeed" member_type: VOTER last_known_addr { host: "127.4.159.188" port: 34189 } } }
I20260807 08:52:01.127779 6898 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae79d9ae8bc246a68cc6feed44f46275 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 08:52:01.127808 6898 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae79d9ae8bc246a68cc6feed44f46275 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "45d8d6472eab48e5a7b79e970e5e1580" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45d8d6472eab48e5a7b79e970e5e1580" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 37535 } } peers { permanent_uuid: "ae79d9ae8bc246a68cc6feed44f46275" member_type: VOTER last_known_addr { host: "127.4.159.189" port: 37101 } } peers { permanent_uuid: "e4dfecedacc248499206c038bcd1eeed" member_type: VOTER last_known_addr { host: "127.4.159.188" port: 34189 } } }
I20260807 08:52:01.127848 6898 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae79d9ae8bc246a68cc6feed44f46275 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 08:52:01.128669 6918 mvcc.cc:205] Tried to move back new op lower bound from 7315835785664806912 to 7315835785552875520. Current Snapshot: MvccSnapshot[applied={T|T < 7315835785568075776}]
W20260807 08:52:01.128881 6915 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ae79d9ae8bc246a68cc6feed44f46275: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 08:52:01.128957 6915 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 08:52:01.139323 6851 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45d8d6472eab48e5a7b79e970e5e1580 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "45d8d6472eab48e5a7b79e970e5e1580" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45d8d6472eab48e5a7b79e970e5e1580" member_type: VOTER last_known_addr { host: "127.4.159.190" port: 37535 } } peers { permanent_uuid: "ae79d9ae8bc246a68cc6feed44f46275" member_type: VOTER last_known_addr { host: "127.4.159.189" port: 37101 } } peers { permanent_uuid: "e4dfecedacc248499206c038bcd1eeed" member_type: VOTER last_known_addr { host: "127.4.159.188" port: 34189 } } }
I20260807 08:52:01.139412 6851 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45d8d6472eab48e5a7b79e970e5e1580 [sys.catalog]: This master's current role is: LEADER
I20260807 08:52:01.142422 6881 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 45d8d6472eab48e5a7b79e970e5e1580: Generated new TSK 0
I20260807 08:52:01.142642 6881 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 08:52:01.194963 6604 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:52:01.193111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45416 (local address 127.4.159.190:37535)
0807 08:52:01.193156 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0807 08:52:01.193158 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:52:01.194225 (+ 1067us) server_negotiation.cc:407] Connection header received
0807 08:52:01.194379 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:52:01.194381 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:52:01.194394 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:52:01.194427 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:52:01.194896 (+ 469us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45416: BlockingRecv error: recv got EOF from 127.0.0.1:45416 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260807 08:52:01.195863 6894 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:52:01.193526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41986 (local address 127.4.159.188:34189)
0807 08:52:01.193566 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0807 08:52:01.193568 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:52:01.195359 (+ 1791us) server_negotiation.cc:407] Connection header received
0807 08:52:01.195451 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:52:01.195452 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:52:01.195463 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:52:01.195489 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:52:01.195807 (+ 318us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41986: BlockingRecv error: recv got EOF from 127.0.0.1:41986 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260807 08:52:01.196645 6721 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:52:01.193882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37064 (local address 127.4.159.189:37101)
0807 08:52:01.193925 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0807 08:52:01.193927 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:52:01.196200 (+ 2273us) server_negotiation.cc:407] Connection header received
0807 08:52:01.196267 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:52:01.196268 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:52:01.196279 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:52:01.196328 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:52:01.196584 (+ 256us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37064: BlockingRecv error: recv got EOF from 127.0.0.1:37064 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260807 08:52:01.198663 6721 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:52:01.197906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37078 (local address 127.4.159.189:37101)
0807 08:52:01.197940 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0807 08:52:01.197941 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 08:52:01.198154 (+ 213us) server_negotiation.cc:407] Connection header received
0807 08:52:01.198248 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:52:01.198249 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:52:01.198266 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:52:01.198293 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:52:01.198621 (+ 328us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37078: BlockingRecv error: recv got EOF from 127.0.0.1:37078 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260807 08:52:01.201869 6721 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:52:01.201047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37086 (local address 127.4.159.189:37101)
0807 08:52:01.201112 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0807 08:52:01.201114 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:52:01.201359 (+ 245us) server_negotiation.cc:407] Connection header received
0807 08:52:01.201447 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:52:01.201448 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:52:01.201458 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:52:01.201485 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:52:01.201826 (+ 341us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37086: BlockingRecv error: recv got EOF from 127.0.0.1:37086 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260807 08:52:01.205768 6721 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:52:01.205040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37100 (local address 127.4.159.189:37101)
0807 08:52:01.205072 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0807 08:52:01.205074 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:52:01.205288 (+ 214us) server_negotiation.cc:407] Connection header received
0807 08:52:01.205377 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:52:01.205378 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:52:01.205388 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:52:01.205415 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:52:01.205728 (+ 313us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37100: BlockingRecv error: recv got EOF from 127.0.0.1:37100 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260807 08:52:01.211735 6721 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:52:01.210931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37102 (local address 127.4.159.189:37101)
0807 08:52:01.210965 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0807 08:52:01.210966 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 08:52:01.211253 (+ 287us) server_negotiation.cc:407] Connection header received
0807 08:52:01.211344 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:52:01.211345 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:52:01.211355 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:52:01.211381 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:52:01.211697 (+ 316us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37102: BlockingRecv error: recv got EOF from 127.0.0.1:37102 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260807 08:52:01.221627 6721 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:52:01.220896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37114 (local address 127.4.159.189:37101)
0807 08:52:01.220928 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0807 08:52:01.220930 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:52:01.221151 (+ 221us) server_negotiation.cc:407] Connection header received
0807 08:52:01.221241 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:52:01.221242 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:52:01.221251 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:52:01.221279 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:52:01.221572 (+ 293us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37114: BlockingRecv error: recv got EOF from 127.0.0.1:37114 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260807 08:52:01.239838 6721 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:52:01.238926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37124 (local address 127.4.159.189:37101)
0807 08:52:01.238964 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0807 08:52:01.238966 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:52:01.239291 (+ 325us) server_negotiation.cc:407] Connection header received
0807 08:52:01.239397 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:52:01.239398 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:52:01.239410 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:52:01.239439 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:52:01.239786 (+ 347us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37124: BlockingRecv error: recv got EOF from 127.0.0.1:37124 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260807 08:52:01.273727 6721 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:52:01.273111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37134 (local address 127.4.159.189:37101)
0807 08:52:01.273151 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0807 08:52:01.273153 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:52:01.273261 (+ 108us) server_negotiation.cc:407] Connection header received
0807 08:52:01.273318 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:52:01.273319 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:52:01.273331 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:52:01.273376 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:52:01.273674 (+ 298us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37134: BlockingRecv error: recv got EOF from 127.0.0.1:37134 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260807 08:52:01.339489 6721 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:52:01.338876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37140 (local address 127.4.159.189:37101)
0807 08:52:01.338945 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0807 08:52:01.338947 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:52:01.339013 (+ 66us) server_negotiation.cc:407] Connection header received
0807 08:52:01.339064 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:52:01.339065 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:52:01.339076 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:52:01.339126 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:52:01.339431 (+ 305us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37140: BlockingRecv error: recv got EOF from 127.0.0.1:37140 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260807 08:52:01.469208 6721 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:52:01.468576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37146 (local address 127.4.159.189:37101)
0807 08:52:01.468653 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0807 08:52:01.468655 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:52:01.468727 (+ 72us) server_negotiation.cc:407] Connection header received
0807 08:52:01.468799 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:52:01.468800 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:52:01.468811 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:52:01.468852 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:52:01.469145 (+ 293us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37146: BlockingRecv error: recv got EOF from 127.0.0.1:37146 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260807 08:52:01.726868 6721 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:52:01.726240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37150 (local address 127.4.159.189:37101)
0807 08:52:01.726294 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0807 08:52:01.726296 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:52:01.726306 (+ 10us) server_negotiation.cc:407] Connection header received
0807 08:52:01.726393 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:52:01.726394 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:52:01.726405 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:52:01.726456 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:52:01.726818 (+ 362us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37150: BlockingRecv error: recv got EOF from 127.0.0.1:37150 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
I20260807 08:52:02.067901 6848 catalog_manager.cc:1259] Loaded cluster ID: 9688cc2f59b74fdb8eda24c64f97d88f
I20260807 08:52:02.067973 6848 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e4dfecedacc248499206c038bcd1eeed: loading cluster ID for follower catalog manager: success
I20260807 08:52:02.068784 6848 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e4dfecedacc248499206c038bcd1eeed: acquiring CA information for follower catalog manager: success
I20260807 08:52:02.069072 6848 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e4dfecedacc248499206c038bcd1eeed: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 08:52:02.129398 6915 catalog_manager.cc:1259] Loaded cluster ID: 9688cc2f59b74fdb8eda24c64f97d88f
I20260807 08:52:02.129470 6915 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ae79d9ae8bc246a68cc6feed44f46275: loading cluster ID for follower catalog manager: success
I20260807 08:52:02.130303 6915 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ae79d9ae8bc246a68cc6feed44f46275: acquiring CA information for follower catalog manager: success
I20260807 08:52:02.130614 6915 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ae79d9ae8bc246a68cc6feed44f46275: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 08:52:02.242493 7021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:52:02.239970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37164 (local address 127.4.159.189:37101)
0807 08:52:02.240085 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0807 08:52:02.240087 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:52:02.240118 (+ 31us) server_negotiation.cc:407] Connection header received
0807 08:52:02.240221 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:52:02.240222 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:52:02.240275 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:52:02.240346 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:52:02.240491 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:52:02.240992 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:52:02.241733 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:52:02.241926 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:52:02.241982 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:52:02.242140 (+ 158us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:52:02.242252 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:52:02.242315 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0807 08:52:02.242385 (+ 70us) server_negotiation.cc:299] Negotiation successful
0807 08:52:02.242405 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92,"thread_start_us":56,"threads_started":1}
I20260807 08:52:02.245980 7031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:52:02.244037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42000 (local address 127.4.159.188:34189)
0807 08:52:02.244127 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0807 08:52:02.244129 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:52:02.244136 (+ 7us) server_negotiation.cc:407] Connection header received
0807 08:52:02.244171 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:52:02.244171 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:52:02.244205 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:52:02.244281 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:52:02.244349 (+ 68us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:52:02.244808 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:52:02.245433 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:52:02.245606 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:52:02.245646 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:52:02.245741 (+ 95us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:52:02.245829 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:52:02.245873 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0807 08:52:02.245889 (+ 16us) server_negotiation.cc:299] Negotiation successful
0807 08:52:02.245904 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73,"thread_start_us":45,"threads_started":1}
I20260807 08:52:02.250717 7021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:52:02.247486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37170 (local address 127.4.159.189:37101)
0807 08:52:02.247540 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0807 08:52:02.247542 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:52:02.247793 (+ 251us) server_negotiation.cc:407] Connection header received
0807 08:52:02.247894 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:52:02.247895 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:52:02.247905 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:52:02.247931 (+ 26us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:52:02.248237 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:52:02.248924 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:52:02.249950 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:52:02.250227 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:52:02.250294 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:52:02.250445 (+ 151us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:52:02.250477 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:52:02.250522 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0807 08:52:02.250631 (+ 109us) server_negotiation.cc:299] Negotiation successful
0807 08:52:02.250650 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260807 08:52:02.250849 7037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:52:02.247518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45418 (local address 127.4.159.190:37535)
0807 08:52:02.247648 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0807 08:52:02.247650 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:52:02.247817 (+ 167us) server_negotiation.cc:407] Connection header received
0807 08:52:02.247868 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:52:02.247869 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:52:02.247919 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:52:02.247995 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:52:02.248156 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:52:02.248853 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:52:02.249796 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:52:02.250119 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:52:02.250179 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:52:02.250445 (+ 266us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:52:02.250556 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:52:02.250609 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0807 08:52:02.250751 (+ 142us) server_negotiation.cc:299] Negotiation successful
0807 08:52:02.250772 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":54,"threads_started":1}
I20260807 08:52:02.252632 7031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:52:02.250301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42014 (local address 127.4.159.188:34189)
0807 08:52:02.250354 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0807 08:52:02.250356 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:52:02.250449 (+ 93us) server_negotiation.cc:407] Connection header received
0807 08:52:02.250524 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:52:02.250525 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:52:02.250536 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:52:02.250584 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:52:02.250754 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:52:02.251363 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:52:02.252175 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:52:02.252353 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:52:02.252406 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:52:02.252496 (+ 90us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:52:02.252516 (+ 20us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:52:02.252544 (+ 28us) server_negotiation.cc:1035] Waiting for connection context
0807 08:52:02.252574 (+ 30us) server_negotiation.cc:299] Negotiation successful
0807 08:52:02.252588 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 08:52:02.255028 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 6499
I20260807 08:52:02.265120 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 6611
I20260807 08:52:02.273526 4734 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQz81E_/build/release/bin/kudu with pid 6724
2026-08-07T08:52:02Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (2910 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (5757 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (268783 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST